diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..98faec3 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,63 @@ +# HydroPlot Architecture Notes + +HydroPlot is organized around five user-facing workflows. New features should land in the workflow where the user decision happens, not in a catch-all tools page. + +## Workflows + +- **Stations** (`overview`): find a stream gage, inspect station map context, live/current condition checks, and record coverage. +- **Site Analysis** (`single-analysis`): one selected gage, hydrographs, duration curves, exports, drought indicators, SPI, and baseflow proxy views. +- **Compare Sites** (`comparisons`): multi-gage overlays, period comparisons, and site relationship analysis. +- **Reach Analysis** (`reach-analysis`): selected upstream/downstream gage pairs, NHD reach map, gain/loss screening, reach plots, baseflow waterfall, and future groundwater observation support. +- **Watershed** (`watershed`): broader basin, HUC, land-cover, dam, flowline, and future well/log context. + +## Groundwater Source Rules + +Groundwater support should be implemented as public-source tiers with explicit eligibility. + +1. **USGS groundwater field measurements** + - Public monitoring data. + - Eligible for time-series screening when enough records exist. + - First implementation target. + +2. **Washington Ecology EIM groundwater data** + - Public Ecology/partner monitoring data when accessed through published search/download/API paths. + - Eligible only after schema and public access are verified. + +3. **Washington Ecology well logs / well reports** + - Public context data, but locations and private-well details can be sensitive or approximate. + - Context only unless a record is clearly a monitoring time series with usable public measurements. + - Do not expose owner, address, parcel, phone/email, or other private fields. + +## Groundwater UI Placement + +- Primary: **Reach Analysis** + - Button-driven "Find public groundwater data" for the selected reach. + - Show monitoring wells separately from context-only well logs. + - Map markers must identify source and eligibility. + - Trend/correlation summaries only for eligible monitoring time series. + +- Secondary: **Watershed** + - Basin-scale well/log inventory context. + - No reach-level interpretation. + +- Later/optional: **Site Analysis** + - Single-gage nearby groundwater context if it supports an actual user workflow. + +## Data And Performance Rules + +- Do not fetch groundwater data automatically on every Streamlit rerun. +- Cache public-data calls by source, reach bounds, dates, and buffer distance. +- Normalize provider output into a safe schema before UI rendering. +- Drop private fields at the data boundary. +- Label approximate locations and context-only sources in the UI. +- Avoid claiming calibrated groundwater modeling; use "screening", "proxy", and "observation support". + +## Verification Pattern + +Each new data source needs: + +- Unit tests for normalization. +- Eligibility tests that prevent context-only records from being analyzed. +- Fixture/mocked tests for missing values, sparse time series, and private-field removal. +- One validation case in `docs/cases/` once the workflow produces a useful output. + diff --git a/docs/cases/_template/README.md b/docs/cases/_template/README.md new file mode 100644 index 0000000..6070bf9 --- /dev/null +++ b/docs/cases/_template/README.md @@ -0,0 +1,35 @@ +--- +case: +title: "" +hydroplot_version: "" +showcases: + - +data_source: "" +runtime_minutes: 0 +created: YYYY-MM-DD +--- + +# + +## Scenario + +Describe the PNW hydrology question, the reach or gage, and why it matters. + +## What This Proves About HydroPlot + +- `<feature>`: describe the reusable HydroPlot analysis behavior being validated. + +## How To Run + +```powershell +python scripts/run_case_study.py docs/cases/<case>/case.yml +``` + +## Outputs + +- `outputs/<file>.csv`: describe table. +- `outputs/<file>.png`: describe figure. + +## Validation + +List the published report, agency method, or expected range used for PASS/FLAG checks. diff --git a/docs/cases/_template/case.yml b/docs/cases/_template/case.yml new file mode 100644 index 0000000..f1a95e9 --- /dev/null +++ b/docs/cases/_template/case.yml @@ -0,0 +1,8 @@ +case: template +site_ids: [] +start_date: "2000-01-01" +end_date: "2025-01-01" +analyses: + - baseflow + - signatures +validation: [] diff --git a/docs/cases/pnw_baseflow_signatures/README.md b/docs/cases/pnw_baseflow_signatures/README.md new file mode 100644 index 0000000..141967f --- /dev/null +++ b/docs/cases/pnw_baseflow_signatures/README.md @@ -0,0 +1,28 @@ +# PNW Baseflow And Hydrologic Signatures + +## Scenario + +This case validates HydroPlot's baseflow and hydrologic signature workflow on a Pacific Northwest daily streamflow record. It focuses on method agreement, plausible BFI range, flow-duration behavior, and flashiness. + +## What This Proves About HydroPlot + +- `lyne_hollick_filter`: bounded multi-pass recursive filter. +- `eckhardt_filter`: independent two-parameter filter. +- `compute_hydrologic_signatures`: compact basin-behavior summary. +- `validate_range`: explicit PASS/FLAG result instead of informal notebook checks. + +## How To Run + +```powershell +python -m scripts docs/cases/pnw_baseflow_signatures/case.yml +``` + +## Outputs + +- `outputs/baseflow_components.csv` +- `outputs/signatures.csv` +- `outputs/validation_summary.csv` + +## Validation + +BFI and flashiness are screened against method-comparison ranges. These checks are not a substitute for a basin-specific groundwater study. diff --git a/docs/cases/pnw_baseflow_signatures/case.yml b/docs/cases/pnw_baseflow_signatures/case.yml new file mode 100644 index 0000000..ec9fa86 --- /dev/null +++ b/docs/cases/pnw_baseflow_signatures/case.yml @@ -0,0 +1,12 @@ +case: pnw_baseflow_signatures +site_ids: + - "12422500" +start_date: "2000-01-01" +end_date: "2025-01-01" +analyses: + - baseflow + - signatures +validation: + - metric: baseflow_index_lh + lower: 0.20 + upper: 0.95 diff --git a/docs/cases/spokane_reach_gain_loss/README.md b/docs/cases/spokane_reach_gain_loss/README.md new file mode 100644 index 0000000..716645b --- /dev/null +++ b/docs/cases/spokane_reach_gain_loss/README.md @@ -0,0 +1,28 @@ +# Spokane Reach Gain/Loss Screening + +## Scenario + +This case validates HydroPlot's reach-scale gain/loss screening on a Spokane River reach where upstream/downstream discharge differences are important for dry-season interpretation. + +## What This Proves About HydroPlot + +- `build_reach_chain`: selected gages can be represented in upstream-to-downstream order. +- `derive_adjacent_reaches`: a river continuum can be assessed reach-by-reach. +- `summarize_reach_gain_loss`: paired upstream/downstream gain-loss calculation. +- Low-flow median gain/loss: dry-window reach gain/loss signal. +- `classify_thermal_sensitivity`: screening context for shade, low flow, and losing/gaining reach conditions. + +## How To Run + +```powershell +python -m scripts docs/cases/spokane_reach_gain_loss/case.yml +``` + +## Outputs + +- `outputs/reach_gain_loss.csv` +- `outputs/validation_summary.csv` + +## Validation + +This is a screening workflow. It does not claim to replace seepage runs, groundwater modeling, TTools, Shade, Heat Source, QUAL2K, or QUAL2Kw. diff --git a/docs/cases/spokane_reach_gain_loss/case.yml b/docs/cases/spokane_reach_gain_loss/case.yml new file mode 100644 index 0000000..b19cd09 --- /dev/null +++ b/docs/cases/spokane_reach_gain_loss/case.yml @@ -0,0 +1,11 @@ +case: spokane_reach_gain_loss +site_ids: + upstream: "12422500" + downstream: "12424000" +start_date: "2015-07-01" +end_date: "2015-09-30" +reach_km: 20 +analyses: + - reach_gain_loss + - thermal_context +validation: [] diff --git a/docs/superpowers/plans/2026-06-05-hydroplot-aquascope-improvement.md b/docs/superpowers/plans/2026-06-05-hydroplot-aquascope-improvement.md new file mode 100644 index 0000000..0fad086 --- /dev/null +++ b/docs/superpowers/plans/2026-06-05-hydroplot-aquascope-improvement.md @@ -0,0 +1,2103 @@ +# HydroPlot AquaScope-Informed Improvement Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Improve HydroPlot by adopting the strongest reproducibility, validation, baseflow, groundwater, reach, and trend ideas from AquaScope-style demos while avoiding notebook-only, duplicated, or non-PNW implementation patterns. + +**Architecture:** Keep HydroPlot as the source of truth. Implement reusable, tested analysis modules first, add PNW validation cases second, then wire proven outputs into the existing Streamlit dashboard. Do not add AquaScope as a runtime dashboard dependency; port useful algorithms and case-study discipline into HydroPlot-native modules. + +**Tech Stack:** Python 3.11, pandas, numpy, scipy, pymannkendall, pytest, Streamlit, Plotly/Matplotlib, existing USGS/NLDI/HyRiver helpers, GitHub feature branch + pull request workflow. + +--- + +## Operating Model + +Work on a feature branch: + +```powershell +git status --short +git switch -c feature/hydroplot-groundwater-reach-validation +``` + +Use this loop for every task: + +```powershell +python -m pytest tests/<focused_test_file>.py -q +git diff +git add <changed files> +git commit -m "<type>: <specific change>" +``` + +Push and open a draft PR after the first two passing task commits: + +```powershell +git push -u origin feature/hydroplot-groundwater-reach-validation +``` + +Best way to see changes as we go: + +- **Code review:** `git diff` after every task, with one focused commit per behavior. +- **Verification:** focused pytest command per task, then broader regression suite after dashboard wiring. +- **Scientific outputs:** generated CSV/PNG/Markdown under `docs/cases/<case>/outputs/`. +- **Dashboard:** local Streamlit run after UI tasks, with screenshots or browser checks for changed pages. +- **GitHub:** draft PR containing task checklist, verification log, and remaining risk notes. + +## Commit Hygiene Rules + +Every commit should look like a deliberate human-authored change: + +- One coherent behavior per commit. Do not mix analysis logic, dashboard UI, docs, and unrelated cleanup in the same commit. +- Tests travel with the behavior they verify. A new algorithm commit includes its focused tests. +- Avoid formatting churn. Do not reformat untouched files or reorder imports globally. +- Avoid broad refactors unless the task explicitly requires moving code to preserve clarity. +- Commit messages should explain the user-visible or developer-visible capability, not the mechanical edit. +- Generated case outputs get their own commit only when they are intentional review artifacts. +- If a task touches more than four files, pause and check whether it should split into two commits. +- Before committing, inspect `git diff --stat` and `git diff` for accidental edits. +- PR description should group commits by capability: baseflow, signatures, changepoints, reach groundwater, validation cases, dashboard wiring. + +Good commit examples: + +```text +feat: add reusable baseflow method comparison +feat: add reach groundwater gain loss summaries +feat: show baseflow method comparison in dashboard +docs: add HydroPlot validation case template +``` + +Bad commit examples: + +```text +update stuff +ai changes +fix tests and dashboard and docs +misc improvements +``` + +## Reach UX Rules + +The reach workflow must not expose topology plumbing as a clunky expert-only process. + +- The primary user action should be: pick an anchor/site or choose a few gages, then review an ordered river chain. +- Show the chain visually and textually as `upstream -> downstream`, with adjacent reach cards/rows. +- Use simple status language: `Verified mainstem`, `Needs review`, `Tributary/diversion possible`, `Not enough data`. +- Put technical details such as NLDI mode, signed distance, COMID, and metadata source behind an expander. +- Do not make users manually enter upstream/downstream order if HydroPlot can infer it from metadata. +- If metadata cannot verify order, let users continue with a clear warning and mark outputs as screening-only. +- For a multi-gage chain, default results should be reach-by-reach rows, not a dense matrix. +- Keep the initial view focused on the decision: which reaches are gaining, losing, neutral, or uncertain during the selected period. +- Avoid page-load network scans. Topology discovery should happen from a clear button such as `Find Related Gages` or `Build Reach Chain`. +- Cache discovered topology and fetched flows for the selected site/date range using existing Streamlit cache patterns. + +## Lean Verification Rules + +Do not add defensive checks everywhere. The code should stay readable and fast. + +- Keep runtime validation at module boundaries: public analysis functions, case runner inputs, and user-triggered dashboard actions. +- Prefer tests over production checks when a condition is a developer contract rather than a likely user/data failure. +- Avoid repeated `isinstance`, schema, and range checks inside loops or vectorized calculations. +- Do not add broad `try/except Exception` around analysis math. Let programming errors fail loudly. +- Use explicit, narrow handling only for expected external failures: network fetch failure, missing site data, empty paired reach data, or insufficient record length. +- Expensive verification belongs in pytest, case-study runners, or on-demand dashboard buttons, not page-load paths. +- Cache or reuse dashboard results through existing Streamlit patterns; do not recompute baseflow, signatures, or changepoints multiple times per rerun. +- Keep PASS/FLAG validation out of core algorithm internals. Algorithms return results; validators inspect results in tests/cases/UI summaries. + +--- + +## Current Assessment + +### AquaScope Strengths To Adopt + +- Reproducible case-study structure with scenario, method, data source, outputs, and validation. +- Flood-frequency validation against published reference values. +- Dual-method baseflow comparison with Lyne-Hollick and Eckhardt filters. +- Hydrologic signatures for quick basin behavior summaries. +- Mann-Kendall trend plus Sen's slope and Pettitt changepoint. +- Output artifacts that make results inspectable outside the app. + +### AquaScope Weaknesses To Improve On + +- Notebook-level fetch/cache code is duplicated across cases. +- Some `dataretrieval` calls shown in demos emit deprecation warnings. +- Broad `except Exception` fallbacks hide failure modes. +- Validation constants live in notebooks instead of reusable configs/tests. +- National examples are useful demos but not tailored to HydroPlot's PNW reach and groundwater purpose. +- Not a dashboard-first implementation. + +### HydroPlot Strengths To Preserve + +- Existing Streamlit dashboard and page module organization. +- Existing `hydrology.analysis.frequency`, `hydrology.analysis.trends`, and `hydrology.analysis.indicators`. +- Existing USGS retry/chunking fetchers. +- Existing PNW/site inventory orientation. +- Existing reach-analysis and NLDI scaffolding. +- Existing pytest suite. + +--- + +## File Structure + +Create: + +- `hydrology/analysis/baseflow.py` + Reusable baseflow filters, result dataclasses, BFI comparison, and method quality flags. + +- `hydrology/analysis/signatures.py` + Hydrologic signatures: flow duration quantiles, flashiness, high/low-flow frequency, seasonality, BFI summary, recession summary hooks. + +- `hydrology/analysis/changepoints.py` + Pettitt changepoint and optional PELT-compatible interface without making heavy dependencies mandatory. + +- `hydrology/analysis/reach_groundwater.py` + Reach-scale gain/loss, normalized contribution, low-flow contribution, classification, and confidence flags. + +- `hydrology/analysis/reach_topology.py` + Small pure helpers for validating station-pair direction, building ordered station chains, and deriving adjacent reach segments from NLDI/navigation metadata. This keeps topology checks separate from gain/loss math. + +- `hydrology/analysis/temperature_context.py` + Lightweight riparian/thermal sensitivity context. This is not QUAL2K; it prepares defensible reach descriptors inspired by TTools/Shade/QUAL2K workflows. Keep this as a simple screening helper, not a large rules engine. + +- `hydrology/analysis/validation.py` + Shared validation result objects and PASS/FLAG helpers for case studies. + +- `hydrology/app/page_modules/groundwater.py` + Dashboard panel for baseflow comparison and reach groundwater summaries, only after analysis modules are verified. + +- `docs/cases/_template/README.md` + HydroPlot-native version of the AquaScope case template. + +- `docs/cases/spokane_groundwater_reach/README.md` + PNW validation case for groundwater/reach behavior. + +- `docs/cases/pnw_baseflow_signatures/README.md` + PNW validation case for baseflow and signatures. + +- `docs/cases/red_river_trend_benchmark/README.md` + Non-PNW benchmark case used only to verify changepoint/trend behavior against a well-documented published example. + +- `scripts/run_case_study.py` + CLI runner for case configs that produces repeatable output artifacts. + +Modify: + +- `hydrology/analysis/__init__.py` + Export new public analysis functions. + +- `hydrology/analysis/indicators.py` + Deprecate or delegate current BFI implementation to `baseflow.py` while keeping backward compatibility. + +- `hydrology/analysis/trends.py` + Add Sen's slope fields if missing and delegate changepoints to `changepoints.py`. + +- `hydrology/analysis/frequency.py` + Add Q-Q/P-P diagnostic table helpers and deterministic bootstrap seed support. + +- `hydrology/app/page_modules/indicators.py` + Replace single-method BFI display with method comparison where available. + +- `hydrology/app/page_modules/reach_analysis.py` + Add reach gain/loss and groundwater contribution summaries. + +- `hydrology/app/page_modules/single_analysis.py` + Add flood-frequency diagnostics and changepoint output in focused sections. + +- `requirements.txt` + Add only lightweight dependencies if needed. Avoid AquaScope as a dependency. + +Test: + +- `tests/test_baseflow.py` +- `tests/test_signatures.py` +- `tests/test_changepoints.py` +- `tests/test_reach_topology.py` +- `tests/test_reach_groundwater.py` +- `tests/test_temperature_context.py` +- `tests/test_validation_cases.py` +- Existing app tests that cover page imports and plot config. + +--- + +## Task 1: Branch And Baseline Verification + +**Files:** +- No code changes. + +- [ ] **Step 1: Check workspace state** + +Run: + +```powershell +git status --short +``` + +Expected: review any existing uncommitted files. Do not overwrite unrelated user changes. + +- [ ] **Step 2: Create feature branch** + +Run: + +```powershell +git switch -c feature/hydroplot-groundwater-reach-validation +``` + +Expected: branch created from current checkout. + +- [ ] **Step 3: Run baseline tests** + +Run: + +```powershell +python -m pytest tests/test_trends.py tests/test_usgs.py tests/test_app_plot_config.py -q +``` + +Expected: PASS or documented existing failures before feature work starts. + +- [ ] **Step 4: Commit plan if not already committed** + +Run: + +```powershell +git add docs/superpowers/plans/2026-06-05-hydroplot-aquascope-improvement.md +git commit -m "docs: plan HydroPlot groundwater and validation upgrades" +``` + +Expected: one docs commit. + +--- + +## Task 2: Baseflow Module With Lyne-Hollick And Eckhardt + +**Files:** +- Create: `hydrology/analysis/baseflow.py` +- Modify: `hydrology/analysis/__init__.py` +- Modify: `hydrology/analysis/indicators.py` +- Test: `tests/test_baseflow.py` + +- [ ] **Step 1: Write failing baseflow tests** + +Create `tests/test_baseflow.py`: + +```python +import numpy as np +import pandas as pd + +from hydrology.analysis.baseflow import ( + BaseflowResult, + compare_baseflow_methods, + eckhardt_filter, + lyne_hollick_filter, +) + + +def _daily_flow(values): + return pd.Series(values, index=pd.date_range("2020-01-01", periods=len(values), freq="D")) + + +def test_lyne_hollick_returns_components_bounded_by_total_flow(): + flow = _daily_flow([100, 120, 180, 140, 110, 90, 95, 105, 130, 115]) + + result = lyne_hollick_filter(flow, alpha=0.925, passes=3) + + assert isinstance(result, BaseflowResult) + assert list(result.components.columns) == ["total_flow", "baseflow", "quickflow"] + assert (result.components["baseflow"] >= 0).all() + assert (result.components["baseflow"] <= result.components["total_flow"]).all() + assert 0 <= result.bfi <= 1 + assert result.method == "lyne_hollick" + + +def test_eckhardt_filter_responds_to_bfi_max_and_stays_bounded(): + flow = _daily_flow([100, 110, 130, 125, 115, 105, 95, 100, 108, 102]) + + conservative = eckhardt_filter(flow, alpha=0.98, bfi_max=0.50) + permissive = eckhardt_filter(flow, alpha=0.98, bfi_max=0.80) + + assert (conservative.components["baseflow"] <= conservative.components["total_flow"]).all() + assert (permissive.components["baseflow"] <= permissive.components["total_flow"]).all() + assert permissive.bfi >= conservative.bfi + assert conservative.method == "eckhardt" + + +def test_compare_baseflow_methods_flags_large_disagreement(): + flow = _daily_flow(np.r_[np.repeat(100.0, 20), 500.0, np.repeat(90.0, 20)]) + + comparison = compare_baseflow_methods(flow) + + assert {"lyne_hollick_bfi", "eckhardt_bfi", "bfi_difference", "agreement"}.issubset(comparison) + assert comparison["agreement"] in {"strong", "moderate", "weak"} + assert comparison["bfi_difference"] >= 0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_baseflow.py -q +``` + +Expected: FAIL because `hydrology.analysis.baseflow` does not exist. + +- [ ] **Step 3: Implement `baseflow.py`** + +Create `hydrology/analysis/baseflow.py`: + +```python +from __future__ import annotations + +from dataclasses import dataclass +from typing import Dict + +import numpy as np +import pandas as pd + +from ..core.logging_setup import get_logger + +logger = get_logger(__name__) + + +@dataclass(frozen=True) +class BaseflowResult: + method: str + components: pd.DataFrame + bfi: float + parameters: Dict[str, float] + + +def _clean_daily_flow(daily_q: pd.Series) -> pd.Series: + series = pd.Series(daily_q).dropna().astype(float) + series = series[np.isfinite(series)] + series = series[series >= 0] + return series.sort_index() + + +def _result(method: str, total: pd.Series, baseflow: np.ndarray, parameters: Dict[str, float]) -> BaseflowResult: + base = np.clip(baseflow.astype(float), 0, total.values.astype(float)) + quick = total.values.astype(float) - base + components = pd.DataFrame( + { + "total_flow": total.values.astype(float), + "baseflow": base, + "quickflow": quick, + }, + index=total.index, + ) + total_sum = float(components["total_flow"].sum()) + bfi = float(components["baseflow"].sum() / total_sum) if total_sum > 0 else float("nan") + return BaseflowResult(method=method, components=components, bfi=bfi, parameters=parameters) + + +def lyne_hollick_filter(daily_q: pd.Series, alpha: float = 0.925, passes: int = 3) -> BaseflowResult: + flow = _clean_daily_flow(daily_q) + if flow.empty: + return _result("lyne_hollick", flow, np.array([], dtype=float), {"alpha": alpha, "passes": passes}) + if not 0 < alpha < 1: + raise ValueError("alpha must be between 0 and 1") + if passes < 1: + raise ValueError("passes must be >= 1") + + q = flow.values.astype(float) + quick = q.copy() + direction = 1 + for _ in range(passes): + work = quick if direction == 1 else quick[::-1] + filtered = np.zeros_like(work) + for i in range(1, len(work)): + filtered[i] = alpha * filtered[i - 1] + ((1 + alpha) / 2.0) * (work[i] - work[i - 1]) + filtered[i] = min(max(filtered[i], 0.0), work[i]) + quick = filtered if direction == 1 else filtered[::-1] + direction *= -1 + + baseflow = q - quick + return _result("lyne_hollick", flow, baseflow, {"alpha": alpha, "passes": float(passes)}) + + +def eckhardt_filter(daily_q: pd.Series, alpha: float = 0.98, bfi_max: float = 0.80) -> BaseflowResult: + flow = _clean_daily_flow(daily_q) + if flow.empty: + return _result("eckhardt", flow, np.array([], dtype=float), {"alpha": alpha, "bfi_max": bfi_max}) + if not 0 < alpha < 1: + raise ValueError("alpha must be between 0 and 1") + if not 0 < bfi_max < 1: + raise ValueError("bfi_max must be between 0 and 1") + + q = flow.values.astype(float) + base = np.zeros_like(q) + base[0] = min(q[0], q[0] * bfi_max) + denominator = 1 - alpha * bfi_max + for i in range(1, len(q)): + numerator = (1 - bfi_max) * alpha * base[i - 1] + (1 - alpha) * bfi_max * q[i] + base[i] = min(q[i], max(0.0, numerator / denominator)) + + return _result("eckhardt", flow, base, {"alpha": alpha, "bfi_max": bfi_max}) + + +def compare_baseflow_methods(daily_q: pd.Series) -> Dict[str, float | str]: + lh = lyne_hollick_filter(daily_q) + ek = eckhardt_filter(daily_q) + diff = abs(lh.bfi - ek.bfi) + if diff <= 0.05: + agreement = "strong" + elif diff <= 0.10: + agreement = "moderate" + else: + agreement = "weak" + return { + "lyne_hollick_bfi": lh.bfi, + "eckhardt_bfi": ek.bfi, + "bfi_difference": diff, + "agreement": agreement, + } +``` + +- [ ] **Step 4: Export and preserve indicator compatibility** + +Modify `hydrology/analysis/__init__.py` to export: + +```python +from .baseflow import BaseflowResult, compare_baseflow_methods, eckhardt_filter, lyne_hollick_filter +``` + +Modify `calculate_baseflow_index_timeseries()` in `hydrology/analysis/indicators.py` to delegate internally: + +```python +from .baseflow import lyne_hollick_filter + +result = lyne_hollick_filter(daily_q, alpha=alpha, passes=1) +df = result.components.copy() +``` + +Keep the rolling BFI logic unchanged after `df` is created. + +- [ ] **Step 5: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_baseflow.py tests/test_app_indicators.py -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +Run: + +```powershell +git add hydrology/analysis/baseflow.py hydrology/analysis/__init__.py hydrology/analysis/indicators.py tests/test_baseflow.py +git commit -m "feat: add reusable baseflow method comparison" +``` + +--- + +## Task 3: Hydrologic Signatures + +**Files:** +- Create: `hydrology/analysis/signatures.py` +- Modify: `hydrology/analysis/__init__.py` +- Test: `tests/test_signatures.py` + +- [ ] **Step 1: Write failing signature tests** + +Create `tests/test_signatures.py`: + +```python +import numpy as np +import pandas as pd + +from hydrology.analysis.signatures import compute_hydrologic_signatures + + +def test_compute_hydrologic_signatures_returns_core_metrics(): + index = pd.date_range("2020-01-01", periods=366, freq="D") + flow = pd.Series(100 + 30 * np.sin(np.linspace(0, 2 * np.pi, 366)), index=index) + + result = compute_hydrologic_signatures(flow) + + assert result["n_days"] == 366 + assert result["mean_flow"] > 0 + assert result["q05"] > result["q50"] > result["q95"] + assert 0 <= result["baseflow_index_lh"] <= 1 + assert result["richards_baker_flashiness"] >= 0 + assert 1 <= result["peak_month"] <= 12 + + +def test_compute_hydrologic_signatures_handles_empty_series(): + result = compute_hydrologic_signatures(pd.Series(dtype=float)) + + assert result == {} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_signatures.py -q +``` + +Expected: FAIL because module does not exist. + +- [ ] **Step 3: Implement signatures** + +Create `hydrology/analysis/signatures.py`: + +```python +from __future__ import annotations + +from typing import Dict + +import numpy as np +import pandas as pd + +from .baseflow import lyne_hollick_filter + + +def compute_hydrologic_signatures(daily_q: pd.Series) -> Dict[str, float]: + flow = pd.Series(daily_q).dropna().astype(float) + flow = flow[np.isfinite(flow)] + flow = flow[flow >= 0].sort_index() + if flow.empty: + return {} + + diffs = flow.diff().abs().dropna() + flashiness = float(diffs.sum() / flow.sum()) if flow.sum() > 0 else float("nan") + monthly = flow.groupby(flow.index.month).mean() if isinstance(flow.index, pd.DatetimeIndex) else pd.Series(dtype=float) + peak_month = int(monthly.idxmax()) if not monthly.empty else 0 + low_month = int(monthly.idxmin()) if not monthly.empty else 0 + lh = lyne_hollick_filter(flow) + + return { + "n_days": float(len(flow)), + "mean_flow": float(flow.mean()), + "median_flow": float(flow.median()), + "min_flow": float(flow.min()), + "max_flow": float(flow.max()), + "q05": float(flow.quantile(0.95)), + "q10": float(flow.quantile(0.90)), + "q50": float(flow.quantile(0.50)), + "q90": float(flow.quantile(0.10)), + "q95": float(flow.quantile(0.05)), + "coefficient_of_variation": float(flow.std(ddof=1) / flow.mean()) if flow.mean() > 0 else float("nan"), + "richards_baker_flashiness": flashiness, + "baseflow_index_lh": float(lh.bfi), + "high_flow_frequency": float((flow > flow.quantile(0.90)).sum() / len(flow)), + "low_flow_frequency": float((flow < flow.quantile(0.10)).sum() / len(flow)), + "peak_month": float(peak_month), + "low_month": float(low_month), + } +``` + +- [ ] **Step 4: Export** + +Modify `hydrology/analysis/__init__.py`: + +```python +from .signatures import compute_hydrologic_signatures +``` + +- [ ] **Step 5: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_signatures.py tests/test_baseflow.py -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +Run: + +```powershell +git add hydrology/analysis/signatures.py hydrology/analysis/__init__.py tests/test_signatures.py +git commit -m "feat: add hydrologic signature metrics" +``` + +--- + +## Task 4: Pettitt Changepoint And Sen Slope + +**Files:** +- Create: `hydrology/analysis/changepoints.py` +- Modify: `hydrology/analysis/trends.py` +- Modify: `hydrology/analysis/__init__.py` +- Test: `tests/test_changepoints.py` +- Test: `tests/test_trends.py` + +- [ ] **Step 1: Write failing changepoint tests** + +Create `tests/test_changepoints.py`: + +```python +import pandas as pd + +from hydrology.analysis.changepoints import pettitt_test +from hydrology.analysis.trends import mann_kendall_test + + +def test_pettitt_detects_known_step_change(): + index = pd.Index(range(1900, 1940), name="year") + values = pd.Series([10] * 20 + [30] * 20, index=index) + + result = pettitt_test(values) + + assert result["change_index"] in {19, 20} + assert result["change_point"] in {1919, 1920} + assert result["p_value"] < 0.05 + assert result["mean_before"] < result["mean_after"] + + +def test_mann_kendall_exposes_sens_slope_when_available(): + values = pd.Series([1, 2, 3, 4, 5, 6]) + + result = mann_kendall_test(values) + + if result is not None: + assert "sens_slope" in result + assert result["sens_slope"] > 0 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_changepoints.py -q +``` + +Expected: FAIL because `changepoints.py` does not exist or `sens_slope` is missing. + +- [ ] **Step 3: Implement Pettitt** + +Create `hydrology/analysis/changepoints.py`: + +```python +from __future__ import annotations + +from typing import Dict, Any + +import numpy as np +import pandas as pd + + +def pettitt_test(series: pd.Series) -> Dict[str, Any]: + clean = pd.Series(series).dropna().sort_index() + n = len(clean) + if n < 6: + return {} + + values = clean.values.astype(float) + ranks = pd.Series(values).rank().values + u = np.array([2 * np.sum(ranks[: k + 1]) - (k + 1) * (n + 1) for k in range(n)]) + k = int(np.argmax(np.abs(u))) + statistic = float(abs(u[k])) + p_value = float(2 * np.exp((-6 * statistic**2) / (n**3 + n**2))) + before = values[: k + 1] + after = values[k + 1 :] + + return { + "change_index": k, + "change_point": clean.index[k].year if hasattr(clean.index[k], "year") else clean.index[k], + "statistic": statistic, + "p_value": min(max(p_value, 0.0), 1.0), + "mean_before": float(np.mean(before)) if len(before) else float("nan"), + "mean_after": float(np.mean(after)) if len(after) else float("nan"), + "n_points": n, + } +``` + +- [ ] **Step 4: Add Sen slope to trend output** + +Modify `mann_kendall_test()` in `hydrology/analysis/trends.py` result dict: + +```python +"sens_slope": getattr(mk_result, "slope", np.nan), +"sens_intercept": getattr(mk_result, "intercept", np.nan), +``` + +- [ ] **Step 5: Export** + +Modify `hydrology/analysis/__init__.py`: + +```python +from .changepoints import pettitt_test +``` + +- [ ] **Step 6: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_changepoints.py tests/test_trends.py -q +``` + +Expected: PASS. + +- [ ] **Step 7: Commit** + +Run: + +```powershell +git add hydrology/analysis/changepoints.py hydrology/analysis/trends.py hydrology/analysis/__init__.py tests/test_changepoints.py tests/test_trends.py +git commit -m "feat: add changepoint and Sen slope trend outputs" +``` + +--- + +## Task 5: Reach Topology, Station Chains, And Pair Validation + +**Files:** +- Create: `hydrology/analysis/reach_topology.py` +- Modify: `hydrology/analysis/__init__.py` +- Test: `tests/test_reach_topology.py` + +**Purpose:** prove HydroPlot can reason about upstream/downstream station pairs and ordered station chains before applying gain/loss math. This task does not call NLDI directly; it validates metadata returned by existing NLDI helpers and dashboard selections. Network-backed NLDI discovery remains in `hydrology/data/nldi.py`. + +**Research basis:** agency reach workflows do not infer gaining/losing conditions from two arbitrary gages. They require a defensible reach definition, station order, flow-period pairing, and caveats for tributaries/diversions/withdrawals. For a river continuum, the correct unit is an ordered chain of monitoring points and the adjacent segments between them. This task creates the lightweight software boundary for that discipline. + +- [ ] **Step 1: Write failing topology tests** + +Create `tests/test_reach_topology.py`: + +```python +from hydrology.analysis.reach_topology import ( + ReachPair, + build_reach_chain, + derive_adjacent_reaches, + classify_pair_direction, + validate_reach_pair, +) + + +def test_classify_pair_direction_accepts_downstream_metadata(): + sites = [ + {"site_id": "up", "direction": "upstream", "distance_km": 8.0}, + {"site_id": "down", "direction": "downstream", "distance_km": 12.0}, + ] + + assert classify_pair_direction("up", "down", sites, origin_site_id="origin") == "ordered" + + +def test_validate_reach_pair_flags_same_station(): + pair = validate_reach_pair("12422500", "12422500", related_sites=[]) + + assert pair.status == "invalid" + assert "same station" in pair.notes[0] + + +def test_validate_reach_pair_flags_unverified_direction(): + pair = validate_reach_pair( + "12422500", + "12424000", + related_sites=[{"site_id": "12424000", "direction": "upstream"}], + ) + + assert isinstance(pair, ReachPair) + assert pair.status == "unverified" + + +def test_build_reach_chain_orders_sites_from_upstream_to_downstream(): + selected = ["A", "B", "C"] + navigation_sites = [ + {"site_id": "C", "direction": "downstream", "distance_km": 20.0}, + {"site_id": "A", "direction": "upstream", "distance_km": 15.0}, + {"site_id": "B", "direction": "upstream", "distance_km": 5.0}, + ] + + chain = build_reach_chain(selected, navigation_sites, origin_site_id="origin") + + assert [station.site_id for station in chain.stations] == ["A", "B", "C"] + assert chain.status == "verified" + + +def test_derive_adjacent_reaches_returns_continuum_pairs(): + selected = ["A", "B", "C"] + navigation_sites = [ + {"site_id": "A", "direction": "upstream", "distance_km": 15.0}, + {"site_id": "B", "direction": "upstream", "distance_km": 5.0}, + {"site_id": "C", "direction": "downstream", "distance_km": 20.0}, + ] + chain = build_reach_chain(selected, navigation_sites, origin_site_id="origin") + + reaches = derive_adjacent_reaches(chain) + + assert [(reach.upstream_site_id, reach.downstream_site_id) for reach in reaches] == [("A", "B"), ("B", "C")] +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_reach_topology.py -q +``` + +Expected: FAIL because `reach_topology.py` does not exist. + +- [ ] **Step 3: Implement topology helpers** + +Create `hydrology/analysis/reach_topology.py`: + +```python +from __future__ import annotations + +from dataclasses import dataclass +from typing import Dict, Iterable, List + + +@dataclass(frozen=True) +class ReachStation: + site_id: str + direction: str + distance_km: float | None + order_key: float + + +@dataclass(frozen=True) +class ReachChain: + stations: List[ReachStation] + status: str + notes: List[str] + + +@dataclass(frozen=True) +class ReachPair: + upstream_site_id: str + downstream_site_id: str + status: str + notes: List[str] + + +def classify_pair_direction( + upstream_site_id: str, + downstream_site_id: str, + related_sites: Iterable[Dict], + origin_site_id: str | None = None, +) -> str: + by_id = {str(site.get("site_id")): site for site in related_sites} + upstream_meta = by_id.get(str(upstream_site_id)) + downstream_meta = by_id.get(str(downstream_site_id)) + + if downstream_meta and downstream_meta.get("direction") == "downstream": + return "ordered" + if upstream_meta and upstream_meta.get("direction") == "upstream": + return "ordered" + if downstream_meta and downstream_meta.get("direction") == "upstream": + return "reversed_or_tributary" + if upstream_meta and upstream_meta.get("direction") == "downstream": + return "reversed_or_tributary" + return "unknown" + + +def build_reach_chain( + selected_site_ids: Iterable[str], + navigation_sites: Iterable[Dict], + origin_site_id: str | None = None, +) -> ReachChain: + selected = [str(site_id) for site_id in selected_site_ids] + by_id = {str(site.get("site_id")): site for site in navigation_sites} + stations: List[ReachStation] = [] + notes: List[str] = [] + + for site_id in selected: + if site_id == origin_site_id: + stations.append(ReachStation(site_id, "origin", 0.0, 0.0)) + continue + + meta = by_id.get(site_id) + if not meta: + notes.append(f"{site_id} not found in navigation metadata") + stations.append(ReachStation(site_id, "unknown", None, float("inf"))) + continue + + direction = str(meta.get("direction", "unknown")) + distance = meta.get("distance_km") + signed_distance = float(distance) if distance is not None else float("inf") + if direction == "upstream": + signed_distance *= -1 + elif direction != "downstream": + signed_distance = float("inf") + + stations.append(ReachStation(site_id, direction, distance, signed_distance)) + + stations.sort(key=lambda station: station.order_key) + status = "verified" if stations and all(station.direction in {"upstream", "origin", "downstream"} for station in stations) else "unverified" + if len(stations) < 2: + status = "invalid" + notes.append("at least two stations are required to define a reach chain") + return ReachChain(stations, status, notes) + + +def derive_adjacent_reaches(chain: ReachChain) -> List[ReachPair]: + reaches: List[ReachPair] = [] + for upstream, downstream in zip(chain.stations, chain.stations[1:]): + reaches.append( + ReachPair( + upstream.site_id, + downstream.site_id, + chain.status, + chain.notes.copy(), + ) + ) + return reaches + + +def validate_reach_pair( + upstream_site_id: str, + downstream_site_id: str, + related_sites: Iterable[Dict], +) -> ReachPair: + if upstream_site_id == downstream_site_id: + return ReachPair(upstream_site_id, downstream_site_id, "invalid", ["same station selected twice"]) + + direction = classify_pair_direction(upstream_site_id, downstream_site_id, related_sites) + if direction == "ordered": + return ReachPair(upstream_site_id, downstream_site_id, "verified", ["NLDI/navigation metadata supports station order"]) + if direction == "reversed_or_tributary": + return ReachPair(upstream_site_id, downstream_site_id, "unverified", ["metadata suggests reversed order or tributary/diversion relationship"]) + return ReachPair(upstream_site_id, downstream_site_id, "unverified", ["station order not verified by navigation metadata"]) +``` + +- [ ] **Step 4: Export** + +Modify `hydrology/analysis/__init__.py`: + +```python +from .reach_topology import ReachChain, ReachPair, ReachStation, build_reach_chain, classify_pair_direction, derive_adjacent_reaches, validate_reach_pair +``` + +- [ ] **Step 5: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_reach_topology.py -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +Run: + +```powershell +git add hydrology/analysis/reach_topology.py hydrology/analysis/__init__.py tests/test_reach_topology.py +git commit -m "feat: add reach station chain validation" +``` + +--- + +## Task 6: Reach Groundwater Gain/Loss Analysis + +**Files:** +- Create: `hydrology/analysis/reach_groundwater.py` +- Modify: `hydrology/analysis/__init__.py` +- Test: `tests/test_reach_groundwater.py` + +- [ ] **Step 1: Write failing reach tests** + +Create `tests/test_reach_groundwater.py`: + +```python +import pandas as pd + +from hydrology.analysis.reach_groundwater import classify_reach_gain_loss, summarize_reach_gain_loss + + +def test_summarize_reach_gain_loss_classifies_gaining_reach(): + index = pd.date_range("2021-08-01", periods=10, freq="D") + upstream = pd.Series([100] * 10, index=index) + downstream = pd.Series([130] * 10, index=index) + + result = summarize_reach_gain_loss(upstream, downstream, reach_km=10, drainage_area_sqmi=100) + + assert result["median_gain_cfs"] == 30 + assert result["median_gain_cfs_per_km"] == 3 + assert result["classification"] == "gaining" + assert result["confidence"] == "high" + + +def test_classify_reach_gain_loss_uses_deadband(): + assert classify_reach_gain_loss(0.2, deadband_cfs=1.0) == "neutral" + assert classify_reach_gain_loss(5.0, deadband_cfs=1.0) == "gaining" + assert classify_reach_gain_loss(-5.0, deadband_cfs=1.0) == "losing" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_reach_groundwater.py -q +``` + +Expected: FAIL because module does not exist. + +- [ ] **Step 3: Implement reach module** + +Create `hydrology/analysis/reach_groundwater.py`: + +```python +from __future__ import annotations + +from typing import Dict, Optional + +import numpy as np +import pandas as pd + + +def classify_reach_gain_loss(median_gain_cfs: float, deadband_cfs: float = 1.0) -> str: + if not np.isfinite(median_gain_cfs): + return "insufficient_data" + if median_gain_cfs > deadband_cfs: + return "gaining" + if median_gain_cfs < -deadband_cfs: + return "losing" + return "neutral" + + +def summarize_reach_gain_loss( + upstream_q: pd.Series, + downstream_q: pd.Series, + reach_km: Optional[float] = None, + drainage_area_sqmi: Optional[float] = None, + low_flow_quantile: float = 0.25, + deadband_cfs: float = 1.0, +) -> Dict[str, float | str]: + upstream = pd.Series(upstream_q).dropna().astype(float) + downstream = pd.Series(downstream_q).dropna().astype(float) + paired = pd.concat({"upstream": upstream, "downstream": downstream}, axis=1).dropna() + if paired.empty: + return {"classification": "insufficient_data", "confidence": "none", "n_days": 0.0} + + paired["gain_cfs"] = paired["downstream"] - paired["upstream"] + low_threshold = paired["upstream"].quantile(low_flow_quantile) + low_flow = paired[paired["upstream"] <= low_threshold] + + median_gain = float(paired["gain_cfs"].median()) + low_flow_gain = float(low_flow["gain_cfs"].median()) if not low_flow.empty else float("nan") + classification = classify_reach_gain_loss(median_gain, deadband_cfs=deadband_cfs) + variability = float(paired["gain_cfs"].std(ddof=1)) if len(paired) > 1 else 0.0 + confidence = "high" if len(paired) >= 7 and variability <= max(abs(median_gain), deadband_cfs) else "moderate" + if len(paired) < 7: + confidence = "low" + + result: Dict[str, float | str] = { + "n_days": float(len(paired)), + "median_gain_cfs": median_gain, + "mean_gain_cfs": float(paired["gain_cfs"].mean()), + "low_flow_median_gain_cfs": low_flow_gain, + "classification": classification, + "confidence": confidence, + } + if reach_km and reach_km > 0: + result["median_gain_cfs_per_km"] = median_gain / reach_km + if drainage_area_sqmi and drainage_area_sqmi > 0: + result["median_gain_cfs_per_sqmi"] = median_gain / drainage_area_sqmi + return result +``` + +- [ ] **Step 4: Export** + +Modify `hydrology/analysis/__init__.py`: + +```python +from .reach_groundwater import classify_reach_gain_loss, summarize_reach_gain_loss +``` + +- [ ] **Step 5: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_reach_groundwater.py -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +Run: + +```powershell +git add hydrology/analysis/reach_groundwater.py hydrology/analysis/__init__.py tests/test_reach_groundwater.py +git commit -m "feat: add reach groundwater gain loss summaries" +``` + +--- + +## Task 7: Lightweight Riparian And Temperature Context + +**Files:** +- Create: `hydrology/analysis/temperature_context.py` +- Modify: `hydrology/analysis/__init__.py` +- Test: `tests/test_temperature_context.py` + +**Scope rule:** keep this task small. It should produce one explicit screening helper with a clear disclaimer. Do not add GIS transect extraction, QUAL2K inputs, TTools emulation, or many nested threshold checks in this phase. + +- [ ] **Step 1: Write failing context tests** + +Create `tests/test_temperature_context.py`: + +```python +from hydrology.analysis.temperature_context import classify_thermal_sensitivity + + +def test_classify_thermal_sensitivity_high_for_wide_unshaded_low_flow_reach(): + result = classify_thermal_sensitivity( + summer_flow_cfs=20, + channel_width_m=12, + canopy_cover_pct=15, + groundwater_gain_cfs=-3, + ) + + assert result["class"] == "high" + assert "low canopy cover" in result["drivers"] + assert "losing reach" in result["drivers"] + + +def test_classify_thermal_sensitivity_lower_for_shaded_gaining_reach(): + result = classify_thermal_sensitivity( + summer_flow_cfs=80, + channel_width_m=4, + canopy_cover_pct=85, + groundwater_gain_cfs=10, + ) + + assert result["class"] in {"low", "moderate"} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_temperature_context.py -q +``` + +Expected: FAIL because module does not exist. + +- [ ] **Step 3: Implement lightweight classifier** + +Create `hydrology/analysis/temperature_context.py`: + +```python +from __future__ import annotations + +from typing import Dict, List + + +def classify_thermal_sensitivity( + summer_flow_cfs: float | None, + channel_width_m: float | None, + canopy_cover_pct: float | None, + groundwater_gain_cfs: float | None, +) -> Dict[str, object]: + score = 0 + drivers: List[str] = [] + + if summer_flow_cfs is not None and summer_flow_cfs < 50: + score += 1 + drivers.append("low summer flow") + if channel_width_m is not None and channel_width_m >= 10: + score += 1 + drivers.append("wide channel") + if canopy_cover_pct is not None and canopy_cover_pct < 40: + score += 2 + drivers.append("low canopy cover") + elif canopy_cover_pct is not None and canopy_cover_pct >= 75: + score -= 1 + drivers.append("high canopy cover") + if groundwater_gain_cfs is not None and groundwater_gain_cfs < -1: + score += 1 + drivers.append("losing reach") + elif groundwater_gain_cfs is not None and groundwater_gain_cfs > 1: + score -= 1 + drivers.append("gaining reach") + + if score >= 3: + label = "high" + elif score >= 1: + label = "moderate" + else: + label = "low" + + return { + "class": label, + "score": score, + "drivers": drivers, + "note": "Screening context only; this is not a QUAL2K, Heat Source, Shade, or TTools model.", + } +``` + +- [ ] **Step 4: Export** + +Modify `hydrology/analysis/__init__.py`: + +```python +from .temperature_context import classify_thermal_sensitivity +``` + +- [ ] **Step 5: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_temperature_context.py -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +Run: + +```powershell +git add hydrology/analysis/temperature_context.py hydrology/analysis/__init__.py tests/test_temperature_context.py +git commit -m "feat: add reach thermal sensitivity context" +``` + +--- + +## Task 8: Flood Frequency Diagnostics And Deterministic CIs + +**Files:** +- Modify: `hydrology/analysis/frequency.py` +- Test: `tests/test_frequency.py` + +- [ ] **Step 1: Write failing frequency tests** + +Create `tests/test_frequency.py`: + +```python +import numpy as np + +from hydrology.analysis.frequency import estimate_return_periods, flood_frequency_diagnostics + + +def test_estimate_return_periods_is_reproducible_with_seed(): + peaks = np.array([100, 120, 130, 150, 180, 220, 260, 300, 360, 420, 500, 620]) + + a = estimate_return_periods(peaks, periods=[10, 50], distribution="lp3", random_seed=42) + b = estimate_return_periods(peaks, periods=[10, 50], distribution="lp3", random_seed=42) + + assert a[["lower_ci", "upper_ci"]].equals(b[["lower_ci", "upper_ci"]]) + + +def test_flood_frequency_diagnostics_returns_plotting_table(): + peaks = np.array([100, 120, 130, 150, 180, 220, 260, 300, 360, 420, 500, 620]) + + diagnostics = flood_frequency_diagnostics(peaks, distribution="lp3") + + assert {"observed_flow_cfs", "fitted_flow_cfs", "exceedance_prob", "return_period"}.issubset(diagnostics.columns) + assert len(diagnostics) == len(peaks) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_frequency.py -q +``` + +Expected: FAIL because `random_seed` and diagnostics are missing. + +- [ ] **Step 3: Add deterministic bootstrap seed** + +Modify `estimate_return_periods()` signature: + +```python +random_seed: int | None = None, +``` + +Replace bootstrap sampling: + +```python +rng = np.random.default_rng(random_seed) +boot_sample = rng.choice(peaks, size=n, replace=True) +``` + +- [ ] **Step 4: Add diagnostics helper** + +Add to `hydrology/analysis/frequency.py`: + +```python +def flood_frequency_diagnostics(peaks: np.ndarray, distribution: str = "lp3") -> pd.DataFrame: + peaks = np.asarray(peaks, dtype=float) + peaks = peaks[np.isfinite(peaks) & (peaks > 0)] + if len(peaks) < 10: + return pd.DataFrame() + + positions = get_plotting_positions(peaks) + fits = fit_flood_frequency(peaks, distributions=[distribution], return_periods=positions["return_period"].tolist()) + if distribution not in fits: + return pd.DataFrame() + + fit = fits[distribution] + rows = [] + for _, row in positions.iterrows(): + rp = float(row["return_period"]) + rows.append( + { + "observed_flow_cfs": float(row["flow_cfs"]), + "fitted_flow_cfs": float(fit.quantiles.get(rp, np.nan)), + "exceedance_prob": float(row["exceedance_prob"]), + "return_period": rp, + "distribution": fit.display_name, + } + ) + return pd.DataFrame(rows) +``` + +- [ ] **Step 5: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_frequency.py -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +Run: + +```powershell +git add hydrology/analysis/frequency.py tests/test_frequency.py +git commit -m "feat: add flood frequency diagnostics" +``` + +--- + +## Task 9: Shared Validation Helpers + +**Files:** +- Create: `hydrology/analysis/validation.py` +- Modify: `hydrology/analysis/__init__.py` +- Test: `tests/test_validation_cases.py` + +**Scope rule:** validation helpers are for case studies and tests. Do not call them inside core baseflow, signatures, changepoint, frequency, or reach algorithms. + +- [ ] **Step 1: Write failing validation tests** + +Create `tests/test_validation_cases.py`: + +```python +from hydrology.analysis.validation import validate_range, validate_relative_error + + +def test_validate_relative_error_passes_within_tolerance(): + result = validate_relative_error("100yr flood", observed=443000, expected=475000, tolerance=0.10) + + assert result.status == "PASS" + assert abs(result.relative_error) < 0.10 + + +def test_validate_range_flags_out_of_range(): + result = validate_range("BFI", value=0.95, lower=0.55, upper=0.85) + + assert result.status == "FLAG" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_validation_cases.py -q +``` + +Expected: FAIL because module does not exist. + +- [ ] **Step 3: Implement validation helpers** + +Create `hydrology/analysis/validation.py`: + +```python +from __future__ import annotations + +from dataclasses import dataclass + + +@dataclass(frozen=True) +class ValidationResult: + metric: str + status: str + value: float + expected: float | str + relative_error: float | None = None + message: str = "" + + +def validate_relative_error(metric: str, observed: float, expected: float, tolerance: float) -> ValidationResult: + rel = (observed - expected) / expected if expected else float("inf") + status = "PASS" if abs(rel) <= tolerance else "FLAG" + return ValidationResult( + metric=metric, + status=status, + value=observed, + expected=expected, + relative_error=rel, + message=f"{metric}: {observed:g} vs {expected:g}, rel error {rel:.3f}, tolerance {tolerance:.3f}", + ) + + +def validate_range(metric: str, value: float, lower: float, upper: float) -> ValidationResult: + status = "PASS" if lower <= value <= upper else "FLAG" + return ValidationResult( + metric=metric, + status=status, + value=value, + expected=f"{lower:g} to {upper:g}", + message=f"{metric}: {value:g}, expected {lower:g} to {upper:g}", + ) +``` + +- [ ] **Step 4: Export** + +Modify `hydrology/analysis/__init__.py`: + +```python +from .validation import ValidationResult, validate_range, validate_relative_error +``` + +- [ ] **Step 5: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_validation_cases.py -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +Run: + +```powershell +git add hydrology/analysis/validation.py hydrology/analysis/__init__.py tests/test_validation_cases.py +git commit -m "feat: add validation helpers for case studies" +``` + +--- + +## Task 10: HydroPlot Case Study Template And Runner + +**Files:** +- Create: `docs/cases/_template/README.md` +- Create: `docs/cases/_template/case.yml` +- Create: `scripts/run_case_study.py` +- Test: `tests/test_validation_cases.py` + +- [ ] **Step 1: Add case template docs** + +Create `docs/cases/_template/README.md`: + +```markdown +--- +case: <slug> +title: "<one-line case title>" +hydroplot_version: "<version or commit>" +showcases: + - <analysis_feature> +data_source: "<dataset, URL, and access date>" +runtime_minutes: 0 +created: YYYY-MM-DD +--- + +# <Title> + +## Scenario + +Describe the PNW hydrology question, the reach or gage, and why it matters. + +## What This Proves About HydroPlot + +- `<feature>`: describe the reusable HydroPlot analysis behavior being validated. + +## How To Run + +```powershell +python scripts/run_case_study.py docs/cases/<case>/case.yml +``` + +## Outputs + +- `outputs/<file>.csv`: describe table. +- `outputs/<file>.png`: describe figure. + +## Validation + +List the published report, agency method, or expected range used for PASS/FLAG checks. +``` + +- [ ] **Step 2: Add case config template** + +Create `docs/cases/_template/case.yml`: + +```yaml +case: template +site_ids: [] +start_date: "2000-01-01" +end_date: "2025-01-01" +analyses: + - baseflow + - signatures +validation: [] +``` + +- [ ] **Step 3: Add minimal runner** + +Create `scripts/run_case_study.py`: + +```python +from __future__ import annotations + +import sys +from pathlib import Path + +import yaml + + +def main() -> int: + if len(sys.argv) != 2: + print("Usage: python scripts/run_case_study.py docs/cases/<case>/case.yml") + return 2 + config_path = Path(sys.argv[1]) + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + output_dir = config_path.parent / "outputs" + output_dir.mkdir(exist_ok=True) + summary = output_dir / "run_summary.md" + summary.write_text( + f"# {config['case']} run summary\n\nConfigured analyses: {', '.join(config.get('analyses', []))}\n", + encoding="utf-8", + ) + print(f"Wrote {summary}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) +``` + +- [ ] **Step 4: Run runner smoke test** + +Run: + +```powershell +python scripts/run_case_study.py docs/cases/_template/case.yml +``` + +Expected: `docs/cases/_template/outputs/run_summary.md` created. + +- [ ] **Step 5: Commit** + +Run: + +```powershell +git add docs/cases/_template scripts/run_case_study.py +git commit -m "docs: add HydroPlot validation case template" +``` + +--- + +## Task 11: PNW Baseflow And Groundwater Validation Cases + +**Files:** +- Create: `docs/cases/pnw_baseflow_signatures/README.md` +- Create: `docs/cases/pnw_baseflow_signatures/case.yml` +- Create: `docs/cases/spokane_groundwater_reach/README.md` +- Create: `docs/cases/spokane_groundwater_reach/case.yml` +- Modify: `scripts/run_case_study.py` +- Test: `tests/test_validation_cases.py` + +- [ ] **Step 1: Create baseflow case doc** + +Create `docs/cases/pnw_baseflow_signatures/README.md` with: + +```markdown +# PNW Baseflow And Hydrologic Signatures + +## Scenario + +This case validates HydroPlot's baseflow and hydrologic signature implementation on a Pacific Northwest streamflow record. It focuses on method agreement, plausible BFI range, flow-duration behavior, and flashiness. + +## What This Proves About HydroPlot + +- `lyne_hollick_filter`: bounded multi-pass recursive filter. +- `eckhardt_filter`: independent two-parameter filter. +- `compute_hydrologic_signatures`: compact basin-behavior summary. +- `validate_range`: explicit PASS/FLAG result instead of informal notebook checks. + +## Outputs + +- `outputs/baseflow_components.csv` +- `outputs/signatures.csv` +- `outputs/validation_summary.csv` + +## Validation + +BFI and flashiness are screened against method-comparison ranges. These checks are not a substitute for a basin-specific groundwater study. +``` + +- [ ] **Step 2: Create Spokane reach case doc** + +Create `docs/cases/spokane_groundwater_reach/README.md` with: + +```markdown +# Spokane Groundwater Reach Screening + +## Scenario + +This case validates HydroPlot's reach-scale groundwater screening on a PNW reach where upstream/downstream discharge differences are important for dry-season interpretation. + +## What This Proves About HydroPlot + +- `summarize_reach_gain_loss`: paired upstream/downstream gain-loss calculation. +- Low-flow median gain/loss: groundwater contribution during dry windows. +- `classify_thermal_sensitivity`: screening context for shade, low flow, and losing/gaining reach conditions. + +## Outputs + +- `outputs/reach_gain_loss.csv` +- `outputs/validation_summary.csv` + +## Validation + +This is a screening workflow. It does not claim to replace seepage runs, groundwater modeling, TTools, Shade, Heat Source, QUAL2K, or QUAL2Kw. +``` + +- [ ] **Step 3: Create case configs** + +Create `docs/cases/pnw_baseflow_signatures/case.yml`: + +```yaml +case: pnw_baseflow_signatures +site_ids: + - "12422500" +start_date: "2000-01-01" +end_date: "2025-01-01" +analyses: + - baseflow + - signatures +validation: + - metric: baseflow_index_lh + lower: 0.20 + upper: 0.95 +``` + +Create `docs/cases/spokane_groundwater_reach/case.yml`: + +```yaml +case: spokane_groundwater_reach +site_ids: + upstream: "12422500" + downstream: "12424000" +start_date: "2015-07-01" +end_date: "2015-09-30" +reach_km: 20 +analyses: + - reach_gain_loss + - thermal_context +validation: [] +``` + +- [ ] **Step 4: Extend runner to produce real outputs** + +Modify `scripts/run_case_study.py` to: + +```python +from hydrology.analysis.baseflow import compare_baseflow_methods, lyne_hollick_filter, eckhardt_filter +from hydrology.analysis.signatures import compute_hydrologic_signatures +from hydrology.analysis.reach_groundwater import summarize_reach_gain_loss +from hydrology.data.usgs import fetch_discharge_data +``` + +For `baseflow`, fetch the first `site_ids` entry, run both filters, and write `baseflow_components.csv`. + +For `signatures`, write `signatures.csv`. + +For `reach_gain_loss`, fetch upstream/downstream and write `reach_gain_loss.csv`. + +Keep network failures explicit: if live fetch fails and no cached input exists, return non-zero with a clear message. + +- [ ] **Step 5: Run case smoke tests** + +Run: + +```powershell +python scripts/run_case_study.py docs/cases/pnw_baseflow_signatures/case.yml +``` + +Expected: outputs written, or explicit network/API failure documented. + +Run: + +```powershell +python scripts/run_case_study.py docs/cases/spokane_groundwater_reach/case.yml +``` + +Expected: outputs written, or explicit network/API failure documented. + +- [ ] **Step 6: Commit** + +Run: + +```powershell +git add docs/cases/pnw_baseflow_signatures docs/cases/spokane_groundwater_reach scripts/run_case_study.py +git commit -m "feat: add PNW validation case studies" +``` + +--- + +## Task 12: Dashboard Integration For Indicators + +**Files:** +- Modify: `hydrology/app/page_modules/indicators.py` +- Test: `tests/test_app_indicators.py` + +**Performance rule:** compute method comparison once per selected site/date range and keep it behind existing Streamlit caching or an explicit user action. Do not recompute both filters for every chart render. + +- [ ] **Step 1: Write or update app test** + +Add to `tests/test_app_indicators.py`: + +```python +def test_indicators_imports_baseflow_comparison(): + from hydrology.analysis.baseflow import compare_baseflow_methods + + assert callable(compare_baseflow_methods) +``` + +- [ ] **Step 2: Run test** + +Run: + +```powershell +python -m pytest tests/test_app_indicators.py -q +``` + +Expected: PASS before UI text changes. + +- [ ] **Step 3: Wire method comparison** + +In `hydrology/app/page_modules/indicators.py`, where BFI is computed, import: + +```python +from hydrology.analysis.baseflow import compare_baseflow_methods +``` + +Display: + +- Lyne-Hollick BFI +- Eckhardt BFI +- difference +- agreement label + +Keep existing rolling BFI chart to avoid disrupting the page. + +- [ ] **Step 4: Run app-focused tests** + +Run: + +```powershell +python -m pytest tests/test_app_indicators.py tests/test_app_plot_config.py -q +``` + +Expected: PASS. + +- [ ] **Step 5: Commit** + +Run: + +```powershell +git add hydrology/app/page_modules/indicators.py tests/test_app_indicators.py +git commit -m "feat: show baseflow method comparison in dashboard" +``` + +--- + +## Task 13: Dashboard Integration For Reach Groundwater + +**Files:** +- Modify: `hydrology/app/page_modules/reach_analysis.py` +- Test: `tests/test_app_shared_conditions.py` or create `tests/test_app_reach_groundwater.py` + +**Performance rule:** run gain/loss summaries on already-fetched paired daily series. Do not trigger extra USGS/NLDI calls solely to populate optional context. + +**UX rule:** this page should present an ordered station chain and adjacent reach summaries. Do not expose a topology-debug table by default. The default output should be readable as: + +```text +12422500 -> 12424000 losing -18 cfs median low confidence +12424000 -> 12424500 gaining +11 cfs median verified mainstem +``` + +Technical metadata belongs in an expander. + +- [ ] **Step 1: Write app import test** + +Create `tests/test_app_reach_groundwater.py`: + +```python +def test_reach_groundwater_helpers_importable(): + from hydrology.analysis.reach_groundwater import summarize_reach_gain_loss + from hydrology.analysis.temperature_context import classify_thermal_sensitivity + + assert callable(summarize_reach_gain_loss) + assert callable(classify_thermal_sensitivity) +``` + +- [ ] **Step 2: Run test** + +Run: + +```powershell +python -m pytest tests/test_app_reach_groundwater.py -q +``` + +Expected: PASS. + +- [ ] **Step 3: Wire reach summary** + +In `hydrology/app/page_modules/reach_analysis.py`, add a compact summary near the upstream/downstream analysis: + +- median gain/loss cfs +- median gain/loss cfs/km when reach length is available +- low-flow median gain/loss +- classification +- confidence +- thermal sensitivity note when canopy/width values are provided or estimated + +Use plain labels and do not claim this is a calibrated groundwater model. + +- [ ] **Step 4: Run app-focused tests** + +Run: + +```powershell +python -m pytest tests/test_app_reach_groundwater.py tests/test_app_shared_conditions.py -q +``` + +Expected: PASS. + +- [ ] **Step 5: Commit** + +Run: + +```powershell +git add hydrology/app/page_modules/reach_analysis.py tests/test_app_reach_groundwater.py +git commit -m "feat: add reach groundwater dashboard summary" +``` + +--- + +## Task 14: Dashboard Integration For Frequency And Trend Diagnostics + +**Files:** +- Modify: `hydrology/app/page_modules/single_analysis.py` +- Test: `tests/test_app_plot_config.py` +- Test: `tests/test_frequency.py` +- Test: `tests/test_changepoints.py` + +**Performance rule:** diagnostics must stay inside the existing on-demand flood/trend workflows. Do not add new page-load computations. + +- [ ] **Step 1: Add imports behind existing analysis actions** + +In `single_analysis.py`, import: + +```python +from hydrology.analysis.changepoints import pettitt_test +from hydrology.analysis.frequency import flood_frequency_diagnostics +``` + +- [ ] **Step 2: Add flood diagnostic table** + +After flood frequency fit succeeds, show a compact diagnostics table with observed flow, fitted flow, exceedance probability, and return period. + +- [ ] **Step 3: Add trend changepoint summary** + +Where annual trend output is available, call `pettitt_test()` and show: + +- change year +- p-value +- mean before +- mean after + +Only show when there are enough annual values. + +- [ ] **Step 4: Run focused tests** + +Run: + +```powershell +python -m pytest tests/test_frequency.py tests/test_changepoints.py tests/test_app_plot_config.py -q +``` + +Expected: PASS. + +- [ ] **Step 5: Commit** + +Run: + +```powershell +git add hydrology/app/page_modules/single_analysis.py +git commit -m "feat: add frequency and changepoint diagnostics to dashboard" +``` + +--- + +## Task 15: Full Verification And Local Dashboard Check + +**Files:** +- No code changes unless fixing issues. + +- [ ] **Step 1: Run focused new tests** + +Run: + +```powershell +python -m pytest tests/test_baseflow.py tests/test_signatures.py tests/test_changepoints.py tests/test_reach_groundwater.py tests/test_temperature_context.py tests/test_frequency.py tests/test_validation_cases.py -q +``` + +Expected: PASS. + +- [ ] **Step 2: Run relevant app tests** + +Run: + +```powershell +python -m pytest tests/test_app_indicators.py tests/test_app_plot_config.py tests/test_app_shared_conditions.py tests/test_app_reach_groundwater.py -q +``` + +Expected: PASS. + +- [ ] **Step 3: Run broader regression** + +Run: + +```powershell +python -m pytest -q +``` + +Expected: PASS or documented unrelated existing failures. + +- [ ] **Step 4: Start dashboard locally** + +Run: + +```powershell +streamlit run hydrology/app/streamlit_app.py +``` + +Expected: app starts and reports local URL. + +- [ ] **Step 5: Manually inspect changed pages** + +Open the local Streamlit URL and inspect: + +- Indicators page: BFI method comparison appears and existing rolling BFI still renders. +- Reach page: gain/loss classification appears for paired sites. +- Single analysis page: flood diagnostics and changepoint summaries appear only with sufficient data. + +- [ ] **Step 6: Commit any verification fixes** + +Run: + +```powershell +git status --short +git diff +git add <fixed files> +git commit -m "fix: stabilize groundwater dashboard verification" +``` + +Expected: only needed if verification found issues. + +--- + +## Task 16: Push And Open GitHub PR + +**Files:** +- No code changes. + +- [ ] **Step 1: Review final diff** + +Run: + +```powershell +git diff main...HEAD --stat +git log --oneline main..HEAD +``` + +Expected: focused commits matching this plan. + +- [ ] **Step 2: Push branch** + +Run: + +```powershell +git push -u origin feature/hydroplot-groundwater-reach-validation +``` + +Expected: branch pushed. + +- [ ] **Step 3: Open draft PR** + +Use GitHub UI or `gh pr create --draft` if authenticated: + +```powershell +gh pr create --draft --title "Improve HydroPlot groundwater, reach, and validation workflows" --body "Adds tested baseflow comparison, hydrologic signatures, changepoints, reach gain/loss summaries, lightweight thermal context, validation case scaffolding, and dashboard integration." +``` + +Expected: draft PR opened against `main`. + +- [ ] **Step 4: Add PR verification checklist** + +PR body should include: + +```markdown +## Verification + +- [ ] `python -m pytest tests/test_baseflow.py tests/test_signatures.py tests/test_changepoints.py tests/test_reach_groundwater.py tests/test_temperature_context.py tests/test_frequency.py tests/test_validation_cases.py -q` +- [ ] `python -m pytest tests/test_app_indicators.py tests/test_app_plot_config.py tests/test_app_shared_conditions.py tests/test_app_reach_groundwater.py -q` +- [ ] `python -m pytest -q` +- [ ] Streamlit dashboard manually checked for Indicators, Reach Analysis, and Single Analysis pages + +## Notes + +This does not add AquaScope as a dependency and does not claim to implement QUAL2K, QUAL2Kw, TTools, Shade, or Heat Source. +``` + +--- + +## Final Verification Matrix + +| Area | Verification | +|---|---| +| Baseflow methods | `tests/test_baseflow.py`, dashboard indicators check | +| Signatures | `tests/test_signatures.py`, PNW case output | +| Changepoints | `tests/test_changepoints.py`, single analysis display | +| Flood diagnostics | `tests/test_frequency.py`, single analysis display | +| Reach groundwater | `tests/test_reach_groundwater.py`, reach page display | +| Thermal context | `tests/test_temperature_context.py`, reach page note | +| Case-study discipline | `tests/test_validation_cases.py`, `docs/cases/*/outputs` | +| Regression | `python -m pytest -q` | +| User-visible app | local Streamlit check | +| GitHub traceability | feature branch, task commits, draft PR checklist | + +## Scope Boundaries + +This plan intentionally does not: + +- Add AquaScope as a dependency. +- Expand HydroPlot to every national gage. +- Implement full QUAL2K/QUAL2Kw. +- Implement TTools GIS transect extraction. +- Claim calibrated groundwater modeling. +- Replace agency QAPP workflows. + +It does create the foundation needed to add those heavier workflows later with clear interfaces and validation. diff --git a/docs/superpowers/plans/2026-06-06-reach-analysis-map-centered-redesign.md b/docs/superpowers/plans/2026-06-06-reach-analysis-map-centered-redesign.md new file mode 100644 index 0000000..63165a3 --- /dev/null +++ b/docs/superpowers/plans/2026-06-06-reach-analysis-map-centered-redesign.md @@ -0,0 +1,592 @@ +# Reach Analysis Map-Centered Redesign Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Rebuild Reach Analysis as a map-centered gage workflow where candidate reach selection, NHD context, inferred reach length, and analysis readiness are visible together. + +**Architecture:** Keep the first pass scoped to `hydrology/app/page_modules/reach_analysis.py` plus focused helper tests. Do not introduce a new frontend framework or long-running runtime checks. Extract small pure helpers only where they make Streamlit layout and selection state easier to test. + +**Tech Stack:** Streamlit, streamlit-folium, Folium/Leaflet, pandas, pytest, existing HydroPlot inventory/NLDI helpers. + +--- + +## File Structure + +- Modify: `hydrology/app/page_modules/reach_analysis.py` + - Add pure helpers for recommended reach pairs, selected-pair state, candidate labels, map creation, and compact summary rows. + - Restructure `show()` so search controls, candidate list, map, and selected reach summary render as one workspace. + - Move plot settings and manual length override into collapsed advanced sections. +- Modify: `tests/test_app_reach_analysis.py` + - Add fast unit tests for recommended pair selection, non-resetting selected pair state, map bounds/key behavior, and user-facing labels. +- No new production dependency. +- No full app navigation rewrite in this pass. + +--- + +### Task 1: Add Reach Candidate Pair Helpers + +**Files:** +- Modify: `hydrology/app/page_modules/reach_analysis.py` +- Test: `tests/test_app_reach_analysis.py` + +- [ ] **Step 1: Write failing tests for pair recommendations** + +Add these imports to `tests/test_app_reach_analysis.py`: + +```python +from hydrology.app.page_modules.reach_analysis import ( + _build_recommended_reach_pairs, + _pair_key, +) +``` + +Add tests: + +```python +def test_build_recommended_reach_pairs_prefers_mainstem_pairs(): + candidates = [ + {"site_id": "anchor", "position": "Anchor", "distance_km": 0.0, "label": "Anchor | anchor | 0.0 km | Anchor"}, + {"site_id": "up", "position": "Upstream", "distance_km": 5.0, "label": "Upstream | up | 5.0 km | Upstream"}, + {"site_id": "trib", "position": "Tributary", "distance_km": 3.0, "label": "Tributary | trib | 3.0 km | Tributary"}, + {"site_id": "down", "position": "Downstream", "distance_km": 8.0, "label": "Downstream | down | 8.0 km | Downstream"}, + ] + + pairs = _build_recommended_reach_pairs("anchor", candidates, max_pairs=5) + + assert pairs[0]["upstream_id"] == "up" + assert pairs[0]["downstream_id"] == "anchor" + assert pairs[1]["upstream_id"] == "anchor" + assert pairs[1]["downstream_id"] == "down" + assert all(pair["upstream_id"] != pair["downstream_id"] for pair in pairs) + assert any(pair["kind"] == "tributary context" for pair in pairs) + + +def test_pair_key_is_stable_and_readable(): + assert _pair_key("12419000", "12422000") == "12419000__12422000" +``` + +- [ ] **Step 2: Run failing tests** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py::test_build_recommended_reach_pairs_prefers_mainstem_pairs tests/test_app_reach_analysis.py::test_pair_key_is_stable_and_readable -q +``` + +Expected: fails because `_build_recommended_reach_pairs` and `_pair_key` are not defined. + +- [ ] **Step 3: Implement minimal helpers** + +Add to `hydrology/app/page_modules/reach_analysis.py` near existing candidate helpers: + +```python +def _pair_key(upstream_id, downstream_id): + """Return a stable selected-reach key.""" + return f"{upstream_id}__{downstream_id}" + + +def _build_recommended_reach_pairs(origin_site_id, candidates, max_pairs=8): + """Build processable upstream/downstream reach pairs for the workspace.""" + origin_site_id = str(origin_site_id) + by_position = {"Upstream": [], "Downstream": [], "Tributary": []} + for candidate in candidates: + site_id = str(candidate.get("site_id", "")) + if not site_id or site_id == origin_site_id: + continue + position = candidate.get("position") + if position in by_position: + by_position[position].append(candidate) + + def distance_value(candidate): + distance = candidate.get("distance_km") + return float(distance) if distance is not None else 9999.0 + + for values in by_position.values(): + values.sort(key=distance_value) + + pairs = [] + for upstream in by_position["Upstream"]: + pairs.append({ + "key": _pair_key(upstream["site_id"], origin_site_id), + "upstream_id": str(upstream["site_id"]), + "downstream_id": origin_site_id, + "label": f'{upstream["site_id"]} -> {origin_site_id}', + "kind": "mainstem upstream", + "distance_km": upstream.get("distance_km"), + }) + for downstream in by_position["Downstream"]: + pairs.append({ + "key": _pair_key(origin_site_id, downstream["site_id"]), + "upstream_id": origin_site_id, + "downstream_id": str(downstream["site_id"]), + "label": f'{origin_site_id} -> {downstream["site_id"]}', + "kind": "mainstem downstream", + "distance_km": downstream.get("distance_km"), + }) + for tributary in by_position["Tributary"]: + pairs.append({ + "key": _pair_key(tributary["site_id"], origin_site_id), + "upstream_id": str(tributary["site_id"]), + "downstream_id": origin_site_id, + "label": f'{tributary["site_id"]} -> {origin_site_id}', + "kind": "tributary context", + "distance_km": tributary.get("distance_km"), + }) + + seen = set() + unique_pairs = [] + for pair in pairs: + if pair["key"] in seen: + continue + seen.add(pair["key"]) + unique_pairs.append(pair) + return unique_pairs[:max_pairs] +``` + +- [ ] **Step 4: Verify tests pass** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py::test_build_recommended_reach_pairs_prefers_mainstem_pairs tests/test_app_reach_analysis.py::test_pair_key_is_stable_and_readable -q +``` + +Expected: `2 passed`. + +- [ ] **Step 5: Commit** + +```bash +git add hydrology/app/page_modules/reach_analysis.py tests/test_app_reach_analysis.py +git commit -m "feat: recommend processable reach pairs" +``` + +--- + +### Task 2: Make Reach Selection State Stable + +**Files:** +- Modify: `hydrology/app/page_modules/reach_analysis.py` +- Test: `tests/test_app_reach_analysis.py` + +- [ ] **Step 1: Write failing tests for selected-pair state** + +Add import: + +```python +from hydrology.app.page_modules.reach_analysis import _resolve_selected_pair_key +``` + +Add tests: + +```python +def test_resolve_selected_pair_key_keeps_valid_existing_selection(): + pairs = [ + {"key": "up__anchor", "upstream_id": "up", "downstream_id": "anchor"}, + {"key": "anchor__down", "upstream_id": "anchor", "downstream_id": "down"}, + ] + session_state = {"reach_selected_pair_key": "anchor__down"} + + assert _resolve_selected_pair_key(pairs, session_state) == "anchor__down" + + +def test_resolve_selected_pair_key_falls_back_to_first_pair(): + pairs = [{"key": "up__anchor", "upstream_id": "up", "downstream_id": "anchor"}] + session_state = {"reach_selected_pair_key": "missing__pair"} + + assert _resolve_selected_pair_key(pairs, session_state) == "up__anchor" + + +def test_resolve_selected_pair_key_handles_no_pairs(): + assert _resolve_selected_pair_key([], {}) is None +``` + +- [ ] **Step 2: Run failing tests** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py::test_resolve_selected_pair_key_keeps_valid_existing_selection tests/test_app_reach_analysis.py::test_resolve_selected_pair_key_falls_back_to_first_pair tests/test_app_reach_analysis.py::test_resolve_selected_pair_key_handles_no_pairs -q +``` + +Expected: fails because `_resolve_selected_pair_key` is not defined. + +- [ ] **Step 3: Implement helper** + +Add near `_pair_key`: + +```python +def _resolve_selected_pair_key(reach_pairs, session_state): + """Keep a selected reach pair if it remains valid; otherwise choose the first available pair.""" + if not reach_pairs: + return None + valid_keys = {pair["key"] for pair in reach_pairs} + current = session_state.get("reach_selected_pair_key") + if current in valid_keys: + return current + return reach_pairs[0]["key"] +``` + +- [ ] **Step 4: Verify tests pass** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py::test_resolve_selected_pair_key_keeps_valid_existing_selection tests/test_app_reach_analysis.py::test_resolve_selected_pair_key_falls_back_to_first_pair tests/test_app_reach_analysis.py::test_resolve_selected_pair_key_handles_no_pairs -q +``` + +Expected: `3 passed`. + +- [ ] **Step 5: Commit** + +```bash +git add hydrology/app/page_modules/reach_analysis.py tests/test_app_reach_analysis.py +git commit -m "fix: keep selected reach pair stable" +``` + +--- + +### Task 3: Extract Map Builder And Fit Bounds Explicitly + +**Files:** +- Modify: `hydrology/app/page_modules/reach_analysis.py` +- Test: `tests/test_app_reach_analysis.py` + +- [ ] **Step 1: Write failing tests for map options and bounds** + +Add import: + +```python +from hydrology.app.page_modules.reach_analysis import _leaflet_fit_bounds_script +``` + +Add test: + +```python +def test_leaflet_fit_bounds_script_targets_selected_bounds(): + script = _leaflet_fit_bounds_script([[47.0, -118.0], [48.0, -117.0]]) + + assert "fitBounds" in script + assert "[[47.0, -118.0], [48.0, -117.0]]" in script + assert "paddingTopLeft" in script + assert "paddingBottomRight" in script +``` + +- [ ] **Step 2: Run failing test** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py::test_leaflet_fit_bounds_script_targets_selected_bounds -q +``` + +Expected: fails because `_leaflet_fit_bounds_script` is not defined. + +- [ ] **Step 3: Implement explicit Leaflet fit helper** + +Add near `_reach_map_component_key`: + +```python +def _leaflet_fit_bounds_script(bounds): + """Return a Folium-compatible script that forces Leaflet to fit selected reach bounds.""" + return ( + "<script>" + "setTimeout(function(){" + "for (const key in window) {" + "const value = window[key];" + "if (value && value.fitBounds && value.eachLayer) {" + f"value.fitBounds({bounds}, {{paddingTopLeft:[24,24], paddingBottomRight:[24,24], maxZoom:13}});" + "}" + "}" + "}, 250);" + "</script>" + ) +``` + +- [ ] **Step 4: Update map rendering code** + +In `show()`, replace the nested `with st.expander("Reach Map", expanded=False):` block with a `render_reach_map` helper call in the main workspace task. For this task only, keep existing code path but add: + +```python +from folium import Element +... +m.fit_bounds(reach_bounds, padding=(24, 24), max_zoom=13) +m.get_root().html.add_child(Element(_leaflet_fit_bounds_script(reach_bounds))) +st_folium( + m, + width=None, + height=460, + returned_objects=[], + key=_reach_map_component_key(upstream_id, downstream_id, reach_bounds), +) +``` + +Also create the map with: + +```python +m = folium.Map( + location=[center_lat, center_lon], + zoom_start=11, + tiles=None, + max_bounds=True, + control_scale=True, +) +``` + +And set the tile layer with: + +```python +folium.TileLayer( + "CartoDB dark_matter", + name="Base map", + no_wrap=True, + detect_retina=True, +).add_to(m) +``` + +- [ ] **Step 5: Verify tests pass** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py::test_leaflet_fit_bounds_script_targets_selected_bounds tests/test_app_reach_analysis.py::test_map_bounds_focuses_on_selected_gage_markers_not_flowline_extent tests/test_app_reach_analysis.py::test_reach_map_component_key_changes_with_selected_pair_and_bounds -q +``` + +Expected: `3 passed`. + +- [ ] **Step 6: Commit** + +```bash +git add hydrology/app/page_modules/reach_analysis.py tests/test_app_reach_analysis.py +git commit -m "fix: force reach map to selected gage bounds" +``` + +--- + +### Task 4: Restructure Reach Analysis Into Map-Centered Workspace + +**Files:** +- Modify: `hydrology/app/page_modules/reach_analysis.py` +- Test: `tests/test_app_reach_analysis.py` + +- [ ] **Step 1: Write failing tests for visible labels** + +Add tests: + +```python +def test_reach_page_source_uses_gage_not_gage(): + source = __import__("inspect").getsource(__import__("hydrology.app.page_modules.reach_analysis", fromlist=["show"])) + + assert "gage" not in source.lower() + assert "gage" in source.lower() + + +def test_reach_page_source_does_not_bury_map_in_expander(): + source = __import__("inspect").getsource(__import__("hydrology.app.page_modules.reach_analysis", fromlist=["show"])) + + assert 'st.expander("Reach Map"' not in source +``` + +- [ ] **Step 2: Run failing tests** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py::test_reach_page_source_uses_gage_not_gage tests/test_app_reach_analysis.py::test_reach_page_source_does_not_bury_map_in_expander -q +``` + +Expected: second test fails until the map is moved out of the expander. + +- [ ] **Step 3: Replace top expanders with compact search row** + +In `show()`, replace: + +```python +with st.expander("Select reach", expanded=True): + ... +``` + +with direct columns: + +```python +search_col1, search_col2, search_col3, search_col4, search_col5 = st.columns([1.1, 2.0, 3.0, 0.9, 1.2]) +with search_col1: + anchor_state = st.selectbox("State", states, key="reach_anchor_state") +with search_col2: + anchor_search = st.text_input( + "Find gage", + placeholder="River, station, or USGS ID...", + key="reach_anchor_search", + ) +... +with search_col5: + include_tributaries = st.toggle("Tributaries", value=True, key="reach_include_tributaries") + find_gages = st.button("Find Reaches", width="stretch", key="reach_find_related") +``` + +Keep the existing `anchor_filtered`, `anchor_options`, `anchor_id`, `anchor_info`, and NLDI discovery logic. + +- [ ] **Step 4: Replace candidate table default with pair selector** + +After `candidate_records` is built, add: + +```python +reach_pairs = _build_recommended_reach_pairs(anchor_id, candidate_records) +selected_pair_key = _resolve_selected_pair_key(reach_pairs, st.session_state) +if selected_pair_key: + st.session_state["reach_selected_pair_key"] = selected_pair_key +selected_pair = next((pair for pair in reach_pairs if pair["key"] == selected_pair_key), None) +``` + +Use a `st.radio` or compact `st.selectbox` in the left workspace column: + +```python +pair_labels = {pair["label"]: pair["key"] for pair in reach_pairs} +selected_pair_label = next(label for label, key in pair_labels.items() if key == selected_pair_key) +chosen_label = st.radio( + "Candidate reaches", + list(pair_labels.keys()), + index=list(pair_labels.keys()).index(selected_pair_label), + key="reach_pair_radio", +) +st.session_state["reach_selected_pair_key"] = pair_labels[chosen_label] +``` + +Then set: + +```python +upstream_id = selected_pair["upstream_id"] +downstream_id = selected_pair["downstream_id"] +``` + +- [ ] **Step 5: Build three-column workspace** + +Replace the separate `Candidate gages`, `Selected reach`, and nested `Reach Map` sections with: + +```python +candidate_col, map_col, summary_col = st.columns([1.05, 2.15, 1.0]) +with candidate_col: + st.subheader("Candidate Reaches") + ... +with map_col: + st.subheader("Reach Map") + ... +with summary_col: + st.subheader("Selected Reach") + ... +``` + +Render network length metric and readiness in the summary column. Keep `manual_reach_km = 0.0` unless advanced override changes it below. + +- [ ] **Step 6: Move lower-priority controls below workspace** + +Create collapsed sections below the workspace: + +```python +with st.expander("Candidate gage details", expanded=False): + ... +with st.expander("Analysis settings", expanded=False): + ... +with st.expander("Advanced reach length override", expanded=False): + ... +``` + +Keep plot defaults as current: `reach_comparison`, `reach_index`, `seasonal_gain_loss`. + +- [ ] **Step 7: Verify tests pass** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py -q +``` + +Expected: all reach app tests pass. + +- [ ] **Step 8: Commit** + +```bash +git add hydrology/app/page_modules/reach_analysis.py tests/test_app_reach_analysis.py +git commit -m "feat: center reach analysis on map workflow" +``` + +--- + +### Task 5: Verify App Behavior And Deploy + +**Files:** +- Modify only if verification finds a bug. + +- [ ] **Step 1: Run focused tests** + +Run: + +```bash +python -m pytest tests/test_app_reach_analysis.py tests/test_reach_gain_loss.py -q +``` + +Expected: all tests pass. + +- [ ] **Step 2: Run related dashboard tests** + +Run: + +```bash +python -m pytest tests/test_app_indicators.py tests/test_frequency.py tests/test_nwm.py -q +``` + +Expected: all tests pass, confirming recent dashboard fixes were not broken. + +- [ ] **Step 3: Run full suite if focused tests pass** + +Run: + +```bash +python -m pytest -q +``` + +Expected: full test suite passes. + +- [ ] **Step 4: Push branch** + +Run: + +```bash +git push +``` + +Expected: branch pushes to origin. + +- [ ] **Step 5: Deploy to Streamlit host** + +Run: + +```bash +ssh cam@192.168.1.126 'cd /home/cam/source/repos/Hydrology && git pull --ff-only' +ssh cam@192.168.1.126 'pkill -f "streamlit run hydrology/app/app.py" || true' +ssh cam@192.168.1.126 'cd /home/cam/source/repos/Hydrology && source venv/bin/activate && setsid streamlit run hydrology/app/app.py --server.headless true --server.address 0.0.0.0 --server.port 8501 > /tmp/hydroplot-streamlit.log 2>&1 < /dev/null &' +ssh cam@192.168.1.126 'curl -s -o /tmp/hydroplot_home.html -w "%{http_code}\n" http://127.0.0.1:8501/' +``` + +Expected: HTTP `200`. + +- [ ] **Step 6: Manual Streamlit verification** + +Open `http://192.168.1.126:8501/` and verify: + +- Search for a Spokane River gage. +- Click Find Reaches with tributaries enabled. +- Candidate Reaches shows processable pairs, not a giant unfiltered NLDI table. +- Selecting a different pair changes only the selected pair; it does not reset both endpoints to the anchor gage. +- Reach Map is visible without opening an expander. +- Reach Map is zoomed into the selected pair, with no repeated-world view. +- Selected reach line is visually stronger than tributary/context flowlines. +- Run Analysis produces the automated reach summary and plots below the workspace. + +- [ ] **Step 7: Commit fixes only if verification finds issues** + +Use targeted commits such as: + +```bash +git add hydrology/app/page_modules/reach_analysis.py tests/test_app_reach_analysis.py +git commit -m "fix: refine reach map workspace behavior" +``` diff --git a/docs/superpowers/specs/2026-06-06-reach-analysis-map-centered-redesign.md b/docs/superpowers/specs/2026-06-06-reach-analysis-map-centered-redesign.md new file mode 100644 index 0000000..ed044b7 --- /dev/null +++ b/docs/superpowers/specs/2026-06-06-reach-analysis-map-centered-redesign.md @@ -0,0 +1,117 @@ +# Reach Analysis Map-Centered Redesign + +Date: 2026-06-06 + +## Purpose + +Reach Analysis should feel like one gage-centered workspace, not a stack of disconnected expanders. The selected reach, processable candidate gages, NHD river context, and analysis readiness must be visible together so a user can understand what they are evaluating before generating plots. + +## Approved Direction + +Use the Map-Centered Workspace layout: + +- A compact top row selects state, searches for one anchor gage, controls network search radius, toggles tributary discovery, and runs candidate discovery. +- The main pane is always visible and centered on the selected reach. +- Processable candidate reaches are shown next to the map as recommended pairs, not as a large default table. +- Selected reach summary sits beside the map with upstream/downstream IDs, inferred network length, data readiness, and a primary Run Analysis button. +- Raw candidate tables, plot descriptions, manual length override, DPI, and layout controls are collapsed behind advanced/detail disclosures. + +## Non-Negotiable UX Requirements + +- Use `gage`, not `gage`, in visible text. +- Do not make the user type reach length for normal cases. Infer network length from NLDI navigation distances when available. Manual length stays advanced-only and only applies when inference fails. +- Do not show unprocessable candidate gages in the primary picker. NLDI sites outside the HydroPlot inventory can be counted or exposed in details, but they should not be default choices. +- Do not bury the reach map inside a collapsed expander. The map is the main workspace. +- The reach map must auto-fit the selected upstream/downstream gage coordinates. It should not open zoomed out far enough to show repeated world tiles. +- The selected mainstem reach should be visually dominant. Tributary and nearby NHD context should be present but secondary. +- The top of the page should not present countless choices before the user understands the selected reach. + +## Layout + +### Top Search Bar + +Replace the current separate "Select reach", "Candidate gages", "Selected reach", and "Run analysis" sections with a compact control row: + +- State selector. +- Single gage search text input. +- Anchor gage selector filtered by state/search. +- Search radius control with a conservative default. +- Tributaries toggle. +- Find Reaches button. + +The search row should be dense and scan-friendly. It should not include plot settings, DPI, manual length, or raw table controls. + +### Main Workspace + +Render a three-column workspace after an anchor gage is selected: + +- Left: candidate reach list. +- Center: NHD reach map. +- Right: selected reach summary and Run Analysis action. + +The candidate list should prefer recommended upstream/downstream pair cards sorted by network position and distance. The anchor gage can be one endpoint. Tributary candidates should appear as context or explicit alternative choices, not silently mixed into mainstem defaults. + +### Reach Map + +The map should be visible by default and should mount outside collapsed Streamlit expanders. + +Map behavior: + +- Fit bounds to the selected upstream/downstream gage coordinates, with enough padding to see reach context. +- Disable repeated world wrapping where supported by the tile layer or map options. +- Draw selected reach flowlines in a strong highlight. +- Draw tributary/context flowlines in a muted secondary style. +- Mark upstream and downstream gages with distinct colors and labels. +- When flowline geometry is unavailable, still fit to the two gage coordinates and clearly show both markers. + +### Advanced Details + +Move lower-priority controls into collapsed disclosures below the workspace: + +- Full candidate gage table. +- Omitted NLDI sites not present in HydroPlot inventory. +- Manual reach length override. +- Plot selection and plot descriptions. +- Export/DPI/layout controls. + +## Data Flow + +1. Load HydroPlot inventory. +2. Filter anchor gage options by state and search text. +3. On Find Reaches, call NLDI related-site discovery for both directions with optional tributaries. +4. Filter discovered sites to processable HydroPlot inventory before primary display. +5. Build candidate reach options from anchor plus processable upstream/downstream/tributary sites. +6. Default selected reach should be a valid processable pair, preferring an upstream-to-anchor or anchor-to-downstream mainstem pair when available. +7. Estimate network length from NLDI signed distances. +8. Render map and selected reach summary before analysis generation. +9. On Run Analysis, fetch paired discharge data, compute gain/loss summary, then render plots and baseflow proxy outputs below the workspace. + +## Verification + +Unit tests should cover helper behavior without adding expensive runtime checks: + +- Candidate filtering excludes non-inventory sites from primary options. +- Default reach selection chooses different upstream/downstream gages when possible. +- Network length inference works from signed NLDI distances. +- Map bounds are computed from selected gage coordinates, not full flowline extent. +- Map component key changes when selected pair or bounds changes. + +Manual verification should be done in Streamlit: + +- Select a Spokane River anchor gage. +- Find reaches with tributaries enabled. +- Confirm only processable primary candidates appear. +- Select different candidate pairs and confirm upstream/downstream do not reset to the anchor. +- Confirm the map remains visible and zooms to the selected reach. +- Confirm selected reach highlight is visually stronger than tributary/context lines. +- Confirm Run Analysis produces the automated summary and plots below the workspace. + +## Out Of Scope For This Pass + +- Full drag-and-drop candidate assignment. +- Global expansion beyond PNW inventory. +- A physically based groundwater model. +- QUAL2K, TTools canopy modeling, or QAPP-derived regulatory workflows. +- Rebuilding all app navigation outside Reach Analysis. + +Those can be separate feature specs after the core Reach Analysis workflow is usable. diff --git a/hydrology/analysis/__init__.py b/hydrology/analysis/__init__.py index cbb272d..e65eb6a 100644 --- a/hydrology/analysis/__init__.py +++ b/hydrology/analysis/__init__.py @@ -4,8 +4,29 @@ from .stage_discharge import fit_powerlaw_rating_curve from .alerts import AlertMonitor, AlertThreshold, Alert, create_flood_alert, create_low_flow_alert from .multisite import MultiSiteAnalyzer, quick_correlation_check, CorrelationResult, LagAnalysisResult -from .frequency import fit_flood_frequency, estimate_return_periods, low_flow_frequency, get_plotting_positions +from .frequency import ( + estimate_return_periods, + fit_flood_frequency, + flood_frequency_diagnostics, + get_plotting_positions, + low_flow_frequency, +) from .indicators import calculate_spi, calculate_sri, classify_drought, calculate_baseflow_index_timeseries +from .baseflow import BaseflowResult, compare_baseflow_methods, eckhardt_filter, lyne_hollick_filter +from .signatures import compute_hydrologic_signatures +from .changepoints import pettitt_test +from .reach_topology import ( + ReachChain, + ReachPair, + ReachStation, + build_reach_chain, + classify_pair_direction, + derive_adjacent_reaches, + validate_reach_pair, +) +from .reach_gain_loss import classify_reach_gain_loss, summarize_reach_gain_loss +from .temperature_context import classify_thermal_sensitivity +from .validation import ValidationResult, validate_range, validate_relative_error __all__ = [ 'calculate_annual_means', @@ -25,6 +46,7 @@ # Frequency analysis 'fit_flood_frequency', 'estimate_return_periods', + 'flood_frequency_diagnostics', 'low_flow_frequency', 'get_plotting_positions', # Drought indicators @@ -32,4 +54,30 @@ 'calculate_sri', 'classify_drought', 'calculate_baseflow_index_timeseries', + # Baseflow separation + 'BaseflowResult', + 'compare_baseflow_methods', + 'eckhardt_filter', + 'lyne_hollick_filter', + # Hydrologic signatures + 'compute_hydrologic_signatures', + # Changepoints + 'pettitt_test', + # Reach topology + 'ReachChain', + 'ReachPair', + 'ReachStation', + 'build_reach_chain', + 'classify_pair_direction', + 'derive_adjacent_reaches', + 'validate_reach_pair', + # Reach gain/loss screening + 'classify_reach_gain_loss', + 'summarize_reach_gain_loss', + # Temperature context + 'classify_thermal_sensitivity', + # Validation helpers + 'ValidationResult', + 'validate_range', + 'validate_relative_error', ] diff --git a/hydrology/analysis/baseflow.py b/hydrology/analysis/baseflow.py new file mode 100644 index 0000000..e61ff38 --- /dev/null +++ b/hydrology/analysis/baseflow.py @@ -0,0 +1,143 @@ +"""Baseflow separation methods for streamflow hydrographs.""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Dict + +import numpy as np +import pandas as pd + + +@dataclass(frozen=True) +class BaseflowResult: + """Baseflow separation result with daily components and summary BFI.""" + + method: str + components: pd.DataFrame + bfi: float + parameters: Dict[str, float] + + +def _clean_daily_flow(daily_q: pd.Series) -> pd.Series: + series = pd.Series(daily_q).dropna().astype(float) + series = series[np.isfinite(series)] + series = series[series >= 0] + return series.sort_index() + + +def _build_result( + method: str, + total: pd.Series, + baseflow: np.ndarray, + parameters: Dict[str, float], +) -> BaseflowResult: + total_values = total.values.astype(float) + base = np.clip(baseflow.astype(float), 0, total_values) + quick = total_values - base + components = pd.DataFrame( + { + "total_flow": total_values, + "baseflow": base, + "quickflow": quick, + }, + index=total.index, + ) + total_sum = float(components["total_flow"].sum()) + bfi = float(components["baseflow"].sum() / total_sum) if total_sum > 0 else float("nan") + return BaseflowResult(method=method, components=components, bfi=bfi, parameters=parameters) + + +def lyne_hollick_filter( + daily_q: pd.Series, + alpha: float = 0.925, + passes: int = 3, +) -> BaseflowResult: + """Separate baseflow with the Lyne-Hollick recursive digital filter.""" + flow = _clean_daily_flow(daily_q) + if flow.empty: + return _build_result( + "lyne_hollick", + flow, + np.array([], dtype=float), + {"alpha": alpha, "passes": float(passes)}, + ) + if not 0 < alpha < 1: + raise ValueError("alpha must be between 0 and 1") + if passes < 1: + raise ValueError("passes must be >= 1") + + q = flow.values.astype(float) + quick = q.copy() + direction = 1 + for _ in range(passes): + work = quick if direction == 1 else quick[::-1] + filtered = np.zeros_like(work) + for i in range(1, len(work)): + filtered[i] = alpha * filtered[i - 1] + ((1 + alpha) / 2.0) * (work[i] - work[i - 1]) + filtered[i] = min(max(filtered[i], 0.0), work[i]) + quick = filtered if direction == 1 else filtered[::-1] + direction *= -1 + + return _build_result( + "lyne_hollick", + flow, + q - quick, + {"alpha": alpha, "passes": float(passes)}, + ) + + +def eckhardt_filter( + daily_q: pd.Series, + alpha: float = 0.98, + bfi_max: float = 0.80, +) -> BaseflowResult: + """Separate baseflow with the Eckhardt two-parameter recursive filter.""" + flow = _clean_daily_flow(daily_q) + if flow.empty: + return _build_result( + "eckhardt", + flow, + np.array([], dtype=float), + {"alpha": alpha, "bfi_max": bfi_max}, + ) + if not 0 < alpha < 1: + raise ValueError("alpha must be between 0 and 1") + if not 0 < bfi_max < 1: + raise ValueError("bfi_max must be between 0 and 1") + + q = flow.values.astype(float) + base = np.zeros_like(q) + base[0] = min(q[0], q[0] * bfi_max) + denominator = 1 - alpha * bfi_max + for i in range(1, len(q)): + numerator = (1 - bfi_max) * alpha * base[i - 1] + (1 - alpha) * bfi_max * q[i] + base[i] = min(q[i], max(0.0, numerator / denominator)) + + return _build_result( + "eckhardt", + flow, + base, + {"alpha": alpha, "bfi_max": bfi_max}, + ) + + +def compare_baseflow_methods(daily_q: pd.Series) -> Dict[str, float | str]: + """Compare Lyne-Hollick and Eckhardt BFI estimates for the same hydrograph.""" + lyne_hollick = lyne_hollick_filter(daily_q) + eckhardt = eckhardt_filter(daily_q) + difference = abs(lyne_hollick.bfi - eckhardt.bfi) + + if difference <= 0.05: + agreement = "strong" + elif difference <= 0.10: + agreement = "moderate" + else: + agreement = "weak" + + return { + "lyne_hollick_bfi": lyne_hollick.bfi, + "eckhardt_bfi": eckhardt.bfi, + "bfi_difference": difference, + "agreement": agreement, + } diff --git a/hydrology/analysis/changepoints.py b/hydrology/analysis/changepoints.py new file mode 100644 index 0000000..0f15fc0 --- /dev/null +++ b/hydrology/analysis/changepoints.py @@ -0,0 +1,36 @@ +"""Non-parametric changepoint tests for hydrologic time series.""" + +from __future__ import annotations + +from typing import Any, Dict + +import numpy as np +import pandas as pd + + +def pettitt_test(series: pd.Series) -> Dict[str, Any]: + """Detect one distributional changepoint using Pettitt's rank test.""" + clean = pd.Series(series).dropna().sort_index() + n = len(clean) + if n < 6: + return {} + + values = clean.values.astype(float) + ranks = pd.Series(values).rank().values + u = np.array([2 * np.sum(ranks[: k + 1]) - (k + 1) * (n + 1) for k in range(n)]) + k = int(np.argmax(np.abs(u))) + statistic = float(abs(u[k])) + p_value = float(2 * np.exp((-6 * statistic**2) / (n**3 + n**2))) + before = values[: k + 1] + after = values[k + 1 :] + change_label = clean.index[k].year if hasattr(clean.index[k], "year") else clean.index[k] + + return { + "change_index": k, + "change_point": change_label, + "statistic": statistic, + "p_value": min(max(p_value, 0.0), 1.0), + "mean_before": float(np.mean(before)) if len(before) else float("nan"), + "mean_after": float(np.mean(after)) if len(after) else float("nan"), + "n_points": n, + } diff --git a/hydrology/analysis/frequency.py b/hydrology/analysis/frequency.py index ae636ea..59e1b13 100644 --- a/hydrology/analysis/frequency.py +++ b/hydrology/analysis/frequency.py @@ -228,6 +228,7 @@ def estimate_return_periods( periods: List[float] = None, distribution: str = 'best', confidence_level: float = 0.95, + random_seed: Optional[int] = None, ) -> pd.DataFrame: """ Estimate return period flows with confidence intervals. @@ -270,8 +271,9 @@ def estimate_return_periods( alpha = 1 - confidence_level bootstrap_quantiles = {rp: [] for rp in periods} + rng = np.random.default_rng(random_seed) for _ in range(n_bootstrap): - boot_sample = np.random.choice(peaks, size=n, replace=True) + boot_sample = rng.choice(peaks, size=n, replace=True) boot_fits = fit_flood_frequency(boot_sample, distributions=[best_name], return_periods=periods) if best_name in boot_fits: for rp in periods: @@ -300,6 +302,44 @@ def estimate_return_periods( return pd.DataFrame(rows) +def flood_frequency_diagnostics(peaks: np.ndarray, distribution: str = "lp3") -> pd.DataFrame: + """ + Build observed-vs-fitted table for flood-frequency diagnostic plots. + + Args: + peaks: Annual peak discharge values + distribution: Distribution to fit for diagnostic quantiles + + Returns: + DataFrame with observed flows, fitted flows, exceedance probability, + return period, and distribution name. + """ + peaks = np.asarray(peaks, dtype=float) + peaks = peaks[np.isfinite(peaks) & (peaks > 0)] + if len(peaks) < 10: + return pd.DataFrame() + + positions = get_plotting_positions(peaks) + periods = positions["return_period"].astype(float).tolist() + fits = fit_flood_frequency(peaks, distributions=[distribution], return_periods=periods) + if distribution not in fits: + return pd.DataFrame() + + fit = fits[distribution] + rows = [] + for _, row in positions.iterrows(): + return_period = float(row["return_period"]) + rows.append({ + "observed_flow_cfs": float(row["flow_cfs"]), + "fitted_flow_cfs": float(fit.quantiles.get(return_period, np.nan)), + "exceedance_prob": float(row["exceedance_prob"]), + "return_period": return_period, + "distribution": fit.display_name, + }) + + return pd.DataFrame(rows) + + def low_flow_frequency( daily_q: pd.Series, durations: List[int] = None, diff --git a/hydrology/analysis/indicators.py b/hydrology/analysis/indicators.py index 8bbd012..666a0e4 100644 --- a/hydrology/analysis/indicators.py +++ b/hydrology/analysis/indicators.py @@ -5,7 +5,7 @@ - SPI (Standardized Precipitation Index): Gamma-fitted precipitation anomalies - SRI (Standardized Runoff Index): Gamma-fitted streamflow anomalies - Drought classification (D0-D4 severity per US Drought Monitor) -- Rolling Baseflow Index (BFI) for groundwater contribution tracking +- Rolling Baseflow Index (BFI) for baseflow-dominance screening These are standard indicators used in drought monitoring networks (NIDIS, USDM). The SRI is particularly relevant for the Spokane dry reach thesis. @@ -22,6 +22,7 @@ import pandas as pd from scipy import stats as sp_stats +from .baseflow import lyne_hollick_filter from ..core.logging_setup import get_logger logger = get_logger(__name__) @@ -275,13 +276,13 @@ def calculate_baseflow_index_timeseries( window_days: int = 90, ) -> pd.DataFrame: """ - Calculate rolling Baseflow Index (BFI) for tracking groundwater contribution. + Calculate rolling Baseflow Index (BFI) for screening baseflow dominance. Uses the Lyne-Hollick recursive digital filter for baseflow separation, then computes BFI as the ratio of baseflow to total flow over a rolling window. Relevant to Spokane dry reach analysis: declining BFI indicates reduced - groundwater contribution (aquifer depletion or diversion effects). + sustained/baseflow-dominated flow (potentially from groundwater, storage, or regulation effects). Args: daily_q: Daily discharge series (datetime index, values in cfs) @@ -296,22 +297,7 @@ def calculate_baseflow_index_timeseries( logger.warning("Insufficient data for BFI timeseries") return pd.DataFrame() - Q = daily_q.values.astype(float) - - # Lyne-Hollick filter - Q_f = np.zeros_like(Q) - for t in range(1, len(Q)): - Q_f[t] = alpha * Q_f[t-1] + (1 + alpha) / 2 * (Q[t] - Q[t-1]) - Q_f[t] = max(0, Q_f[t]) - - baseflow = np.clip(Q - Q_f, 0, Q) - quickflow = Q - baseflow - - df = pd.DataFrame({ - 'total_flow': Q, - 'baseflow': baseflow, - 'quickflow': quickflow, - }, index=daily_q.index) + df = lyne_hollick_filter(daily_q, alpha=alpha, passes=1).components.copy() # Rolling BFI rolling_total = df['total_flow'].rolling(window=window_days, min_periods=window_days//2).sum() diff --git a/hydrology/analysis/reach_gain_loss.py b/hydrology/analysis/reach_gain_loss.py new file mode 100644 index 0000000..7cc9c7e --- /dev/null +++ b/hydrology/analysis/reach_gain_loss.py @@ -0,0 +1,66 @@ +"""Reach-scale gain/loss summaries for paired streamflow gages.""" + +from __future__ import annotations + +from typing import Dict + +import numpy as np +import pandas as pd + + +def classify_reach_gain_loss(median_gain_cfs: float, deadband_cfs: float = 1.0) -> str: + """Classify reach gain/loss using a small deadband around zero.""" + if not np.isfinite(median_gain_cfs): + return "insufficient_data" + if median_gain_cfs > deadband_cfs: + return "gaining" + if median_gain_cfs < -deadband_cfs: + return "losing" + return "neutral" + + +def summarize_reach_gain_loss( + upstream_q: pd.Series, + downstream_q: pd.Series, + reach_km: float | None = None, + drainage_area_sqmi: float | None = None, + low_flow_quantile: float = 0.25, + deadband_cfs: float = 1.0, +) -> Dict[str, float | str]: + """Summarize paired upstream/downstream flow differences for one reach.""" + upstream = pd.Series(upstream_q).dropna().astype(float) + downstream = pd.Series(downstream_q).dropna().astype(float) + paired = pd.concat({"upstream": upstream, "downstream": downstream}, axis=1).dropna() + if paired.empty: + return {"classification": "insufficient_data", "confidence": "none", "n_days": 0.0} + + paired["gain_cfs"] = paired["downstream"] - paired["upstream"] + low_threshold = paired["upstream"].quantile(low_flow_quantile) + low_flow = paired[paired["upstream"] <= low_threshold] + + median_gain = float(paired["gain_cfs"].median()) + low_flow_gain = float(low_flow["gain_cfs"].median()) if not low_flow.empty else float("nan") + classification = classify_reach_gain_loss(median_gain, deadband_cfs=deadband_cfs) + variability = float(paired["gain_cfs"].std(ddof=1)) if len(paired) > 1 else 0.0 + + if len(paired) < 7: + confidence = "low" + elif variability <= max(abs(median_gain), deadband_cfs): + confidence = "high" + else: + confidence = "moderate" + + result: Dict[str, float | str] = { + "n_days": float(len(paired)), + "median_gain_cfs": median_gain, + "mean_gain_cfs": float(paired["gain_cfs"].mean()), + "low_flow_median_gain_cfs": low_flow_gain, + "classification": classification, + "confidence": confidence, + } + if reach_km and reach_km > 0: + result["median_gain_cfs_per_km"] = median_gain / reach_km + if drainage_area_sqmi and drainage_area_sqmi > 0: + result["median_gain_cfs_per_sqmi"] = median_gain / drainage_area_sqmi + + return result diff --git a/hydrology/analysis/reach_topology.py b/hydrology/analysis/reach_topology.py new file mode 100644 index 0000000..34a7272 --- /dev/null +++ b/hydrology/analysis/reach_topology.py @@ -0,0 +1,155 @@ +"""Reach station ordering helpers built from navigation metadata.""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Dict, Iterable, List + + +@dataclass(frozen=True) +class ReachStation: + """A gage positioned relative to a reach-chain origin.""" + + site_id: str + direction: str + distance_km: float | None + order_key: float + + +@dataclass(frozen=True) +class ReachChain: + """Ordered upstream-to-downstream gage chain.""" + + stations: List[ReachStation] + status: str + notes: List[str] + + +@dataclass(frozen=True) +class ReachPair: + """Adjacent upstream/downstream gage pair.""" + + upstream_site_id: str + downstream_site_id: str + status: str + notes: List[str] + + +def classify_pair_direction( + upstream_site_id: str, + downstream_site_id: str, + related_sites: Iterable[Dict], + origin_site_id: str | None = None, +) -> str: + """Classify whether navigation metadata supports a proposed pair order.""" + by_id = {str(site.get("site_id")): site for site in related_sites} + upstream_meta = by_id.get(str(upstream_site_id)) + downstream_meta = by_id.get(str(downstream_site_id)) + + if downstream_meta and downstream_meta.get("direction") == "downstream": + return "ordered" + if upstream_meta and upstream_meta.get("direction") == "upstream": + return "ordered" + if downstream_meta and downstream_meta.get("direction") == "upstream": + return "reversed_or_tributary" + if upstream_meta and upstream_meta.get("direction") == "downstream": + return "reversed_or_tributary" + return "unknown" + + +def build_reach_chain( + selected_site_ids: Iterable[str], + navigation_sites: Iterable[Dict], + origin_site_id: str | None = None, +) -> ReachChain: + """Order selected sites from upstream to downstream using navigation metadata.""" + selected = [str(site_id) for site_id in selected_site_ids] + by_id = {str(site.get("site_id")): site for site in navigation_sites} + stations: List[ReachStation] = [] + notes: List[str] = [] + + for site_id in selected: + if site_id == origin_site_id: + stations.append(ReachStation(site_id, "origin", 0.0, 0.0)) + continue + + meta = by_id.get(site_id) + if not meta: + notes.append(f"{site_id} not found in navigation metadata") + stations.append(ReachStation(site_id, "unknown", None, float("inf"))) + continue + + direction = str(meta.get("direction", "unknown")) + distance = meta.get("distance_km") + signed_distance = float(distance) if distance is not None else float("inf") + if direction == "upstream": + signed_distance *= -1 + elif direction != "downstream": + signed_distance = float("inf") + + stations.append(ReachStation(site_id, direction, distance, signed_distance)) + + stations.sort(key=lambda station: station.order_key) + verified_directions = {"upstream", "origin", "downstream"} + status = ( + "verified" + if stations and all(station.direction in verified_directions for station in stations) + else "unverified" + ) + if len(stations) < 2: + status = "invalid" + notes.append("at least two stations are required to define a reach chain") + + return ReachChain(stations, status, notes) + + +def derive_adjacent_reaches(chain: ReachChain) -> List[ReachPair]: + """Convert an ordered gage chain into adjacent reach segments.""" + reaches: List[ReachPair] = [] + for upstream, downstream in zip(chain.stations, chain.stations[1:]): + reaches.append( + ReachPair( + upstream.site_id, + downstream.site_id, + chain.status, + chain.notes.copy(), + ) + ) + return reaches + + +def validate_reach_pair( + upstream_site_id: str, + downstream_site_id: str, + related_sites: Iterable[Dict], +) -> ReachPair: + """Validate a proposed upstream/downstream gage pair.""" + if upstream_site_id == downstream_site_id: + return ReachPair( + upstream_site_id, + downstream_site_id, + "invalid", + ["same station selected twice"], + ) + + direction = classify_pair_direction(upstream_site_id, downstream_site_id, related_sites) + if direction == "ordered": + return ReachPair( + upstream_site_id, + downstream_site_id, + "verified", + ["NLDI/navigation metadata supports station order"], + ) + if direction == "reversed_or_tributary": + return ReachPair( + upstream_site_id, + downstream_site_id, + "unverified", + ["metadata suggests reversed order or tributary/diversion relationship"], + ) + return ReachPair( + upstream_site_id, + downstream_site_id, + "unverified", + ["station order not verified by navigation metadata"], + ) diff --git a/hydrology/analysis/signatures.py b/hydrology/analysis/signatures.py new file mode 100644 index 0000000..51f1337 --- /dev/null +++ b/hydrology/analysis/signatures.py @@ -0,0 +1,52 @@ +"""Compact hydrologic signature metrics for daily streamflow.""" + +from __future__ import annotations + +from typing import Dict + +import numpy as np +import pandas as pd + +from .baseflow import lyne_hollick_filter + + +def compute_hydrologic_signatures(daily_q: pd.Series) -> Dict[str, float]: + """Compute a small, dashboard-friendly set of daily-flow signatures.""" + flow = pd.Series(daily_q).dropna().astype(float) + flow = flow[np.isfinite(flow)] + flow = flow[flow >= 0].sort_index() + if flow.empty: + return {} + + total_flow = float(flow.sum()) + flashiness = float(flow.diff().abs().dropna().sum() / total_flow) if total_flow > 0 else float("nan") + if isinstance(flow.index, pd.DatetimeIndex): + monthly = flow.groupby(flow.index.month).mean() + peak_month = int(monthly.idxmax()) + low_month = int(monthly.idxmin()) + else: + peak_month = 0 + low_month = 0 + + baseflow = lyne_hollick_filter(flow) + mean_flow = float(flow.mean()) + + return { + "n_days": float(len(flow)), + "mean_flow": mean_flow, + "median_flow": float(flow.median()), + "min_flow": float(flow.min()), + "max_flow": float(flow.max()), + "q05": float(flow.quantile(0.95)), + "q10": float(flow.quantile(0.90)), + "q50": float(flow.quantile(0.50)), + "q90": float(flow.quantile(0.10)), + "q95": float(flow.quantile(0.05)), + "coefficient_of_variation": float(flow.std(ddof=1) / mean_flow) if mean_flow > 0 else float("nan"), + "richards_baker_flashiness": flashiness, + "baseflow_index_lh": float(baseflow.bfi), + "high_flow_frequency": float((flow > flow.quantile(0.90)).sum() / len(flow)), + "low_flow_frequency": float((flow < flow.quantile(0.10)).sum() / len(flow)), + "peak_month": float(peak_month), + "low_month": float(low_month), + } diff --git a/hydrology/analysis/stage_discharge.py b/hydrology/analysis/stage_discharge.py index 3858b60..f6814a8 100644 --- a/hydrology/analysis/stage_discharge.py +++ b/hydrology/analysis/stage_discharge.py @@ -184,6 +184,199 @@ def offset_powerlaw(H, A, B, H0): return (A, B, 0.0, R2, Q_pred) +def fit_best_rating_curve( + stage: pd.Series, + discharge: pd.Series, + min_points: int = 10, +) -> Dict[str, Any]: + """ + Fit simple and offset power-law ratings; return the better model by R². + + Many USGS gages (e.g. Walla Walla nr Touchet 14018500) have a non-zero + control stage H0. Fitting Q = A * H^B on raw stage then produces garbage + (even negative R²). Prefer Q = A * (H - H0)^B when it improves fit. + + Returns dict with keys: + model: 'offset_powerlaw' | 'powerlaw' | 'none' + A, B, H0, R2, Q_pred, n_points, equation + """ + df = pd.DataFrame({"H": stage, "Q": discharge}) + df = df.replace([np.inf, -np.inf], np.nan).dropna() + df = df[(df["H"] > 0) & (df["Q"] > 0)] + n = len(df) + + empty = { + "model": "none", + "A": np.nan, + "B": np.nan, + "H0": 0.0, + "R2": np.nan, + "Q_pred": pd.Series(index=stage.index, dtype=float), + "n_points": n, + "equation": "n/a", + } + if n < min_points: + logger.warning("Insufficient pairs for rating curve: %s", n) + return empty + + A_s, B_s, R2_s, Q_s = fit_powerlaw_rating_curve( + df["H"], df["Q"], min_points=min_points + ) + A_o, B_o, H0, R2_o, Q_o = fit_offset_powerlaw( + df["H"], df["Q"], min_points=min_points + ) + + # Prefer offset when R² is clearly better (or simple fit is invalid) + use_offset = ( + np.isfinite(R2_o) + and (not np.isfinite(R2_s) or R2_o >= R2_s + 0.02 or R2_s < 0.5) + and np.isfinite(A_o) + and np.isfinite(B_o) + ) + + if use_offset: + A_f, B_f, H0_f, R2_f, Q_full = fit_offset_powerlaw( + stage, discharge, min_points=min_points + ) + eq = f"Q = {A_f:.4g} · (H − {H0_f:.3f})^{B_f:.3f}" + return { + "model": "offset_powerlaw", + "A": float(A_f), + "B": float(B_f), + "H0": float(H0_f), + "R2": float(R2_f) if np.isfinite(R2_f) else np.nan, + "Q_pred": Q_full, + "n_points": n, + "equation": eq, + } + + A_f, B_f, R2_f, Q_full = fit_powerlaw_rating_curve( + stage, discharge, min_points=min_points + ) + eq = f"Q = {A_f:.4g} · H^{B_f:.3f}" + return { + "model": "powerlaw", + "A": float(A_f) if np.isfinite(A_f) else np.nan, + "B": float(B_f) if np.isfinite(B_f) else np.nan, + "H0": 0.0, + "R2": float(R2_f) if np.isfinite(R2_f) else np.nan, + "Q_pred": Q_full, + "n_points": n, + "equation": eq, + } + + +def season_labels(index: pd.DatetimeIndex) -> pd.Series: + """Map datetime index → meteorological season labels (DJF/MAM/JJA/SON).""" + month = pd.DatetimeIndex(index).month + labels = np.full(len(month), "UNK", dtype=object) + labels[np.isin(month, [12, 1, 2])] = "DJF" + labels[np.isin(month, [3, 4, 5])] = "MAM" + labels[np.isin(month, [6, 7, 8])] = "JJA" + labels[np.isin(month, [9, 10, 11])] = "SON" + return pd.Series(labels, index=index, name="season") + + +def predict_rating_discharge( + stage: np.ndarray | pd.Series, + A: float, + B: float, + H0: float = 0.0, +) -> np.ndarray: + """Q = A * (H - H0)^B with safe floor on effective stage.""" + H = np.asarray(stage, dtype=float) + H_eff = np.maximum(H - float(H0), 1e-10) + A = float(A) + B = float(B) + if not np.isfinite(A) or not np.isfinite(B) or A <= 0: + return np.full_like(H, np.nan, dtype=float) + return A * (H_eff ** B) + + +def evaluate_rating_params( + stage: pd.Series | np.ndarray, + discharge: pd.Series | np.ndarray, + A: float, + B: float, + H0: float = 0.0, +) -> Dict[str, Any]: + """ + Score a user- or auto-specified rating Q = A*(H-H0)^B against paired data. + + Returns R², residual stats, equation string, and predicted Q on the clean pairs. + """ + df = pd.DataFrame({"H": stage, "Q": discharge}) + df = df.replace([np.inf, -np.inf], np.nan).dropna() + df = df[(df["H"] > float(H0) + 1e-9) & (df["Q"] > 0)] + + empty = { + "model": "offset_powerlaw" if abs(float(H0)) > 1e-9 else "powerlaw", + "A": float(A), + "B": float(B), + "H0": float(H0), + "R2": np.nan, + "rmse": np.nan, + "mae": np.nan, + "mape": np.nan, + "bias": np.nan, + "nash_sutcliffe": np.nan, + "n_points": 0, + "equation": "n/a", + "Q_pred": np.array([], dtype=float), + "H_clean": np.array([], dtype=float), + "Q_clean": np.array([], dtype=float), + "residuals": np.array([], dtype=float), + } + if len(df) < 3 or not np.isfinite(A) or not np.isfinite(B) or A <= 0: + return empty + + H = df["H"].values + Q = df["Q"].values + Q_hat = predict_rating_discharge(H, A, B, H0) + valid = np.isfinite(Q_hat) + H, Q, Q_hat = H[valid], Q[valid], Q_hat[valid] + if len(Q) < 3: + return empty + + ss_res = float(np.sum((Q - Q_hat) ** 2)) + ss_tot = float(np.sum((Q - Q.mean()) ** 2)) + R2 = 1.0 - ss_res / ss_tot if ss_tot > 0 else np.nan + residuals = Q_hat - Q + abs_err = np.abs(residuals) + rmse = float(np.sqrt(np.mean(residuals ** 2))) + mae = float(np.mean(abs_err)) + nz = Q != 0 + mape = float(np.mean(np.abs(residuals[nz] / Q[nz])) * 100) if np.any(nz) else np.nan + bias = float(np.mean(residuals)) + nse = 1.0 - ss_res / ss_tot if ss_tot > 0 else np.nan + + if abs(float(H0)) > 1e-6: + eq = f"Q = {A:.4g} · (H − {H0:.3f})^{B:.3f}" + model = "offset_powerlaw" + else: + eq = f"Q = {A:.4g} · H^{B:.3f}" + model = "powerlaw" + + return { + "model": model, + "A": float(A), + "B": float(B), + "H0": float(H0), + "R2": float(R2) if np.isfinite(R2) else np.nan, + "rmse": rmse, + "mae": mae, + "mape": mape, + "bias": bias, + "nash_sutcliffe": float(nse) if np.isfinite(nse) else np.nan, + "n_points": int(len(Q)), + "equation": eq, + "Q_pred": Q_hat, + "H_clean": H, + "Q_clean": Q, + "residuals": residuals, + } + + def calculate_residuals( observed: pd.Series, predicted: pd.Series diff --git a/hydrology/analysis/temperature_context.py b/hydrology/analysis/temperature_context.py new file mode 100644 index 0000000..3a858cb --- /dev/null +++ b/hydrology/analysis/temperature_context.py @@ -0,0 +1,49 @@ +"""Lightweight reach thermal-sensitivity screening context.""" + +from __future__ import annotations + +from typing import Dict, List + + +def classify_thermal_sensitivity( + summer_flow_cfs: float | None, + channel_width_m: float | None, + canopy_cover_pct: float | None, + reach_gain_cfs: float | None, +) -> Dict[str, object]: + """Classify simple thermal-sensitivity drivers for a reach.""" + score = 0 + drivers: List[str] = [] + + if summer_flow_cfs is not None and summer_flow_cfs < 50: + score += 1 + drivers.append("low summer flow") + if channel_width_m is not None and channel_width_m >= 10: + score += 1 + drivers.append("wide channel") + if canopy_cover_pct is not None and canopy_cover_pct < 40: + score += 2 + drivers.append("low canopy cover") + elif canopy_cover_pct is not None and canopy_cover_pct >= 75: + score -= 1 + drivers.append("high canopy cover") + if reach_gain_cfs is not None and reach_gain_cfs < -1: + score += 1 + drivers.append("losing reach") + elif reach_gain_cfs is not None and reach_gain_cfs > 1: + score -= 1 + drivers.append("gaining reach") + + if score >= 3: + label = "high" + elif score >= 1: + label = "moderate" + else: + label = "low" + + return { + "class": label, + "score": score, + "drivers": drivers, + "note": "Screening context only; this is not a QUAL2K, Heat Source, Shade, or TTools model.", + } diff --git a/hydrology/analysis/trends.py b/hydrology/analysis/trends.py index c7bb2ad..47a9bf9 100644 --- a/hydrology/analysis/trends.py +++ b/hydrology/analysis/trends.py @@ -58,7 +58,7 @@ def calculate_annual_means( return None # Resample to annual (year start) and calculate mean - annual = df[column].resample('AS').mean().dropna() + annual = df[column].resample('YS').mean().dropna() logger.info(f"Calculated annual means: {column} ({len(annual)} years)") return annual @@ -261,6 +261,8 @@ def mann_kendall_test( 'tau': mk_result.tau, 's': mk_result.s, 'z': mk_result.z, + 'sens_slope': getattr(mk_result, 'slope', np.nan), + 'sens_intercept': getattr(mk_result, 'intercept', np.nan), 'n_points': len(series_clean), 'alpha': alpha } diff --git a/hydrology/analysis/validation.py b/hydrology/analysis/validation.py new file mode 100644 index 0000000..b61bd42 --- /dev/null +++ b/hydrology/analysis/validation.py @@ -0,0 +1,51 @@ +"""Validation helpers for case studies and tests.""" + +from __future__ import annotations + +from dataclasses import dataclass + + +@dataclass(frozen=True) +class ValidationResult: + """Result from a simple PASS/FLAG case-study validation check.""" + + metric: str + status: str + value: float + expected: float | str + relative_error: float | None = None + message: str = "" + + +def validate_relative_error( + metric: str, + observed: float, + expected: float, + tolerance: float, +) -> ValidationResult: + """Validate a value against a reference using relative error.""" + relative_error = (observed - expected) / expected if expected else float("inf") + status = "PASS" if abs(relative_error) <= tolerance else "FLAG" + return ValidationResult( + metric=metric, + status=status, + value=observed, + expected=expected, + relative_error=relative_error, + message=( + f"{metric}: {observed:g} vs {expected:g}, " + f"relative error {relative_error:.3f}, tolerance {tolerance:.3f}" + ), + ) + + +def validate_range(metric: str, value: float, lower: float, upper: float) -> ValidationResult: + """Validate a value against an inclusive expected range.""" + status = "PASS" if lower <= value <= upper else "FLAG" + return ValidationResult( + metric=metric, + status=status, + value=value, + expected=f"{lower:g} to {upper:g}", + message=f"{metric}: {value:g}, expected {lower:g} to {upper:g}", + ) diff --git a/hydrology/app/app.py b/hydrology/app/app.py index 879f5c3..2b8d5cd 100644 --- a/hydrology/app/app.py +++ b/hydrology/app/app.py @@ -12,21 +12,87 @@ import streamlit as st -st.set_page_config(page_title="Hydrology Analysis", page_icon="\U0001f4a7", layout="wide") +st.set_page_config( + page_title="HydroPlot", + page_icon="\U0001f4a7", + layout="wide", + initial_sidebar_state="collapsed", +) -from hydrology.app.styles import apply_custom_css, render_footer, render_dashboard_hero, render_workflow_strip +from hydrology.app.styles import ( + apply_custom_css, + render_footer, + render_dashboard_hero, + render_dashboard_meta, + render_main_nav, +) from hydrology.app.shared import get_inventory from hydrology.visualization.plots import AVAILABLE_PLOTS # Page imports -from hydrology.app.page_modules import overview, single_analysis, comparisons, reach_analysis, alerts, advanced, indicators +from hydrology.app.page_modules import ( + overview, + single_analysis, + comparisons, + reach_analysis, + watershed, +) apply_custom_css() render_dashboard_hero( - "HydroPlot analysis dashboard", - "Explore stations, analyze site behavior, compare records, and monitor climate-linked flow signals from one workspace.", + "HydroPlot", + "Find gages, analyze one site, compare records, evaluate reaches, and inspect basin context — one refined workspace.", +) + +# Define pages with grouping +_single_analysis_page = st.Page( + single_analysis.show, title="Site Analysis", icon="📈", url_path="single-analysis" +) +_overview_page = st.Page( + overview.show, title="Stations", icon="\U0001f4ca", default=True, url_path="overview" +) +_comparisons_page = st.Page( + comparisons.show, title="Compare Sites", icon="\U0001f504", url_path="comparisons" +) +_reach_page = st.Page( + reach_analysis.show, title="Reach Analysis", icon="\U0001f30a", url_path="reach-analysis" +) +_watershed_page = st.Page( + watershed.show, title="Watershed", icon="\U0001f5fa\ufe0f", url_path="watershed" ) -render_workflow_strip() + +pages = { + "Workflows": [ + _overview_page, + _single_analysis_page, + _comparisons_page, + _reach_page, + _watershed_page, + ], +} + +# Hide Streamlit's default multipage chrome; custom main-nav is the product UI. +try: + pg = st.navigation(pages, position="hidden") +except TypeError: + # Older Streamlit without position= support + pg = st.navigation(pages) + +# Active pill highlight for custom main nav +try: + active = getattr(pg, "url_path", None) or getattr(pg, "_url_path", None) or "overview" + st.session_state["_hydro_active_page"] = str(active).strip("/") or "overview" +except Exception: + st.session_state["_hydro_active_page"] = "overview" + +render_main_nav(st.session_state.get("_hydro_active_page")) + +# Store page refs for cross-page navigation +st.session_state["_page_single_analysis"] = _single_analysis_page + +inventory_df = get_inventory() +site_count = len(inventory_df) if inventory_df is not None and not inventory_df.empty else 0 +render_dashboard_meta(site_count=site_count, plot_count=len(AVAILABLE_PLOTS)) # Background cache warmup — pre-fetch data so first user doesn't wait if "warmup_started" not in st.session_state: @@ -37,54 +103,27 @@ def _warmup(): try: from hydrology.data.usgs import fetch_current_conditions, fetch_daily_values from hydrology.data.usgs import DEFAULT_PARAM_DISCHARGE - - # Warm priority sites from datetime import datetime, timedelta + priority = ["12422500", "12424000", "12419000"] fetch_current_conditions(priority) - end = datetime.now().strftime('%Y-%m-%d') - start = (datetime.now() - timedelta(days=365)).strftime('%Y-%m-%d') + end = datetime.now().strftime("%Y-%m-%d") + start = (datetime.now() - timedelta(days=365)).strftime("%Y-%m-%d") for sid in priority: try: - fetch_daily_values(sid, param_cd=DEFAULT_PARAM_DISCHARGE, - start_date=start, end_date=end) + fetch_daily_values( + sid, + param_cd=DEFAULT_PARAM_DISCHARGE, + start_date=start, + end_date=end, + ) except Exception: pass except Exception: - pass # Warmup is best-effort + pass threading.Thread(target=_warmup, daemon=True).start() -# Define pages with grouping -_single_analysis_page = st.Page(single_analysis.show, title="Single Analysis", icon="📈", url_path="single-analysis") - -pages = { - "Dashboard": [ - st.Page(overview.show, title="Overview", icon="\U0001f4ca", default=True, url_path="overview"), - _single_analysis_page, - ], - "Compare": [ - st.Page(comparisons.show, title="Comparisons", icon="\U0001f504", url_path="comparisons"), - st.Page(reach_analysis.show, title="Reach Analysis", icon="\U0001f30a", url_path="reach-analysis"), - ], - "Monitor": [ - st.Page(alerts.show, title="Alerts", icon="\U0001f6a8", url_path="alerts"), - st.Page(indicators.show, title="Indicators", icon="\U0001f321\ufe0f", url_path="indicators"), - st.Page(advanced.show, title="Advanced", icon="\U0001f52c", url_path="advanced"), - ], -} - -pg = st.navigation(pages) - -# Store page refs for cross-page navigation -st.session_state["_page_single_analysis"] = _single_analysis_page - -# Footer in sidebar -inventory_df = get_inventory() -st.sidebar.markdown("---") -site_count = len(inventory_df) if not inventory_df.empty else 0 -st.sidebar.caption(f"Sites: {site_count} | Plots: {len(AVAILABLE_PLOTS)}") - pg.run() render_footer() diff --git a/hydrology/app/interpretation.py b/hydrology/app/interpretation.py index a93b3b7..6fe1e24 100644 --- a/hydrology/app/interpretation.py +++ b/hydrology/app/interpretation.py @@ -3,7 +3,9 @@ from __future__ import annotations from dataclasses import dataclass +from typing import Iterable +import numpy as np import pandas as pd @@ -13,6 +15,7 @@ class InsightCard: value: str body: str state: str = "ready" + help: str = "" def _flow_column(df: pd.DataFrame) -> str | None: @@ -22,6 +25,16 @@ def _flow_column(df: pd.DataFrame) -> str | None: return df.columns[0] if len(df.columns) else None +def _series(df: pd.DataFrame | None, col: str | None = None) -> pd.Series: + if df is None or df.empty: + return pd.Series(dtype=float) + use = col or _flow_column(df) + if not use or use not in df.columns: + return pd.Series(dtype=float) + s = pd.to_numeric(df[use], errors="coerce").dropna() + return s[s >= 0] + + def _status_from_percentile(percentile: float | None) -> tuple[str, str]: if percentile is None: return "Unknown", "limited" @@ -45,6 +58,7 @@ def summarize_flow_context(df: pd.DataFrame | None) -> list[InsightCard]: "No data", "Daily discharge was not available for this site and date range.", "blocked", + help="Widen the date range or choose a gage with continuous daily discharge (00060).", ) ] @@ -60,6 +74,7 @@ def summarize_flow_context(df: pd.DataFrame | None) -> list[InsightCard]: "No values", "The selected discharge column contains no numeric values.", "blocked", + help="The table loaded but discharge cells are empty or non-numeric for this period.", ) ] @@ -91,24 +106,41 @@ def summarize_flow_context(df: pd.DataFrame | None) -> list[InsightCard]: f"{latest_value:,.0f} cfs", f"Latest daily value on {latest_date:%Y-%m-%d}.", "ready", + help=( + "Most recent daily mean discharge in the selected window. " + "Compare with Seasonal Context — the same number can be high or low " + "depending on the time of year." + ), ), InsightCard( "Seasonal Context", f"{percentile:.0f}th pct." if percentile is not None else "Unknown", f"{status} for this time of year based on the selected historical record.", status_state, + help=( + "Percentile of current flow among same calendar days (±15 days) in this " + "record. 50th ≈ typical for the season; 90th ≈ much wetter than normal." + ), ), InsightCard( "Recent Direction", trend_label, trend_body, trend_state, + help=( + "Compares the last 30-day mean to the prior 30 days. Short-term only — " + "not a multi-year trend. Rising after storms is normal." + ), ), InsightCard( "Record Depth", f"{record_years:.1f} yrs", "Enough for trend and duration views." if record_years >= 10 else "Use caution for climate normals and frequency-style interpretation.", "ready" if record_years >= 10 else "limited", + help=( + "Years of daily data in the selected range. Flood frequency and climate " + "normals usually want 10–30+ years; short windows are fine for screening." + ), ), ] @@ -121,6 +153,10 @@ def summarize_recommendations(has_stage: bool, has_climate: bool, record_years: "Flow duration", "Start with flow duration and seasonal anomaly to understand normal vs unusual flow states.", "ready", + help=( + "The FDC and Q10/Q50/Q90 strip show how often high vs low flows occur. " + "Scroll to Interactive Charts → Duration statistics." + ), ) ] if has_climate: @@ -130,6 +166,10 @@ def summarize_recommendations(has_stage: bool, has_climate: bool, record_years: "Use SPI", "Climate data is available, so precipitation and drought context are worth reviewing.", "ready", + help=( + "SPI tracks meteorological drought; SRI tracks runoff drought. " + "Open Drought & Baseflow Indicators below the charts when ready." + ), ) ) else: @@ -139,6 +179,10 @@ def summarize_recommendations(has_stage: bool, has_climate: bool, record_years: "Limited", "Climate-linked plots may be incomplete until weather data can be merged.", "limited", + help=( + "No reliable precip/temp merge for this period. SPI may still fetch " + "Open-Meteo, but co-plotted climate overlays will be thin." + ), ) ) if has_stage: @@ -148,6 +192,10 @@ def summarize_recommendations(has_stage: bool, has_climate: bool, record_years: "Available", "Stage overlay and rating-curve checks are valid for this selected period.", "ready", + help=( + "Gage height is present. Use dual-axis stage on the hydrograph and the " + "Rating curve workshop to tune A, B, and H₀." + ), ) ) if record_years >= 10: @@ -157,6 +205,10 @@ def summarize_recommendations(has_stage: bool, has_climate: bool, record_years: "On demand", "Peak-flow frequency can be run separately without slowing the page load.", "limited", + help=( + "Annual-max flood frequency is opt-in so page load stays fast. " + "Open Frequency Analysis under Advanced when you need return periods." + ), ) ) return cards @@ -179,3 +231,773 @@ def describe_standardized_index(value: float | None) -> tuple[str, str]: if value < 2: return "Very wet signal", "Values above 1.3 are materially wetter than normal." return "Exceptional wet signal", "Values above 2 are rare and indicate very wet or high-flow conditions." + + +def _percentile_label(p: float) -> str: + if p >= 90: + return "much higher than usual" + if p >= 75: + return "higher than usual" + if p >= 25: + return "near the middle of the record" + if p >= 10: + return "lower than usual" + return "much lower than usual" + + +def summarize_flow_duration(df_q: pd.DataFrame | None) -> str: + """Plain-language flow-duration / regime summary from daily Q.""" + s = _series(df_q) + if s.empty or len(s) < 30: + return "Not enough discharge values to characterize flow duration." + + q10 = float(np.percentile(s, 90)) # high flow exceeded ~10% of time + q50 = float(np.percentile(s, 50)) + q90 = float(np.percentile(s, 10)) # low flow exceeded ~90% of time + mean = float(s.mean()) + # Flashiness-ish: high/low ratio + ratio = q10 / max(q90, 1e-6) + if ratio >= 50: + regime = "flashy / high-contrast" + regime_note = ( + "High flows dwarf low flows (Q10/Q90 is large), so the stream swings " + "between wet-season peaks and sustained low baseflow." + ) + elif ratio >= 15: + regime = "moderately variable" + regime_note = ( + "There is a clear high-flow season and a quieter low-flow season, " + "but extremes are not extreme for all rivers." + ) + else: + regime = "relatively steady" + regime_note = ( + "High and low ends of the duration curve are closer together, " + "suggesting more regulated or baseflow-supported behavior." + ) + + return ( + f"**Flow duration / regime ({regime}):** " + f"median daily flow is about **{q50:,.0f} cfs**, with high-flow (Q10) near " + f"**{q10:,.0f} cfs** and low-flow (Q90) near **{q90:,.0f} cfs** " + f"(mean **{mean:,.0f} cfs**). {regime_note}" + ) + + +def summarize_seasonal_pattern(df_q: pd.DataFrame | None) -> str: + """Seasonal mean-flow contrast from daily Q.""" + s = _series(df_q) + if s.empty or len(s) < 90: + return "Not enough data for a seasonal pattern summary." + + months = s.index.month + buckets = { + "winter (DJF)": s[np.isin(months, [12, 1, 2])], + "spring (MAM)": s[np.isin(months, [3, 4, 5])], + "summer (JJA)": s[np.isin(months, [6, 7, 8])], + "fall (SON)": s[np.isin(months, [9, 10, 11])], + } + means = {k: float(v.mean()) for k, v in buckets.items() if len(v) >= 10} + if len(means) < 2: + return "Seasonal contrast could not be estimated for this period." + + wettest = max(means, key=means.get) + driest = min(means, key=means.get) + wet_v, dry_v = means[wettest], means[driest] + factor = wet_v / max(dry_v, 1e-6) + return ( + f"**Seasonal pattern:** average daily flow is highest in **{wettest}** " + f"(~{wet_v:,.0f} cfs) and lowest in **{driest}** (~{dry_v:,.0f} cfs) — " + f"about **{factor:.1f}×** higher in the wetter season. " + "Use monthly boxplots / heatmaps to see timing of the high-flow pulse." + ) + + +def summarize_climate_merged(df_merged: pd.DataFrame | None) -> str: + """Climate linkage when precip/temp are merged to discharge.""" + if df_merged is None or df_merged.empty: + return ( + "**Climate:** no merged temperature/precipitation for this site and period " + "(Open-Meteo/Meteostat may still work for SPI separately)." + ) + + parts = [] + if "Precip_mm" in df_merged.columns: + p = pd.to_numeric(df_merged["Precip_mm"], errors="coerce").dropna() + if not p.empty: + annual = p.resample("YE").sum() + mean_ann = float(annual.mean()) if len(annual) else float(p.sum() / max(len(p) / 365.25, 1)) + wet_days = float((p > 1.0).mean() * 100) + parts.append( + f"mean annual precip ~**{mean_ann:,.0f} mm**, " + f"with precip >1 mm on about **{wet_days:.0f}%** of days" + ) + if "Temp_C" in df_merged.columns: + t = pd.to_numeric(df_merged["Temp_C"], errors="coerce").dropna() + if not t.empty: + parts.append( + f"mean temp **{float(t.mean()):.1f}°C** " + f"(range {float(t.min()):.1f}–{float(t.max()):.1f}°C)" + ) + if not parts: + return "**Climate:** merged frame present but precip/temp columns were empty." + return "**Climate (merged to streamflow days):** " + "; ".join(parts) + "." + + +def summarize_rating_curve(df_q: pd.DataFrame | None) -> str: + """Text summary of stage–discharge fit when gage height is present.""" + if df_q is None or df_q.empty: + return "" + if "Gage_Height_ft" not in df_q.columns or "Discharge_cfs" not in df_q.columns: + return "" + try: + from hydrology.analysis.stage_discharge import fit_best_rating_curve + except Exception: + return "" + + stage = pd.to_numeric(df_q["Gage_Height_ft"], errors="coerce") + q = pd.to_numeric(df_q["Discharge_cfs"], errors="coerce") + fit = fit_best_rating_curve(stage, q, min_points=10) + if fit.get("model") == "none" or not np.isfinite(fit.get("R2", np.nan)): + return ( + "**Rating curve:** stage is present but a stable Q–H fit could not be " + "estimated (need enough positive pairs)." + ) + r2 = fit["R2"] + quality = "strong" if r2 >= 0.9 else "moderate" if r2 >= 0.7 else "weak" + note = "" + if fit["model"] == "offset_powerlaw": + note = ( + f" An offset stage H₀≈{fit['H0']:.2f} ft was needed — common when gage " + "zero is not the zero-flow control elevation." + ) + return ( + f"**Stage–discharge rating:** best model is `{fit['equation']}` " + f"with **R²={r2:.3f}** ({quality} fit, n={fit['n_points']:,}).{note} " + "Points often show seasonal loops (rising vs falling limb / vegetation / ice)." + ) + + +def summarize_selected_plots(plot_keys: Iterable[str]) -> str: + """Explain what the selected plot set is good for.""" + keys = [str(k) for k in plot_keys] + if not keys: + return "No static plots were selected." + + themes = [] + if any(k in keys for k in ("timeseries", "anomaly", "monthly_boxplot", "discharge_heatmap")): + themes.append("time patterns and seasonality") + if any(k in keys for k in ("flow_duration", "low_flow_trend", "7q10_analysis")): + themes.append("low-flow / duration behavior") + if any(k in keys for k in ("flood_frequency", "annual_trend")): + themes.append("peaks and long-term change") + if any(k in keys for k in ("rating_curve",)): + themes.append("stage–discharge relationship") + if any("precip" in k or "temp" in k or "climate" in k or "lag" in k or "hexbin" in k for k in keys): + themes.append("climate–streamflow links") + if not themes: + themes.append("general site diagnostics") + + pretty = ", ".join(keys[:8]) + ("…" if len(keys) > 8 else "") + return ( + f"**Plots generated ({len(keys)}):** {pretty}. " + f"Together they emphasize {', '.join(themes)}." + ) + + +def build_plot_analysis_report( + *, + site_id: str, + site_desc: str, + plot_keys: list[str] | tuple[str, ...], + df_q: pd.DataFrame | None, + df_merged: pd.DataFrame | None = None, +) -> str: + """ + Full markdown narrative after static plot generation. + + Combines record stats, duration/regime, seasons, climate, and rating notes + so the user gets a text explanation next to the figure grid. + """ + s = _series(df_q) + lines: list[str] = [ + f"### Automated read — {site_desc}", + f"USGS **{site_id}** · generated from the selected period and plot set.", + "", + ] + + if s.empty: + lines.append("No usable discharge series was available for this explanation.") + return "\n".join(lines) + + years = (s.index.max() - s.index.min()).days / 365.25 + latest = float(s.iloc[-1]) + latest_date = s.index.max() + # Seasonal percentile for latest + doy = latest_date.dayofyear + seasonal = s[(s.index.dayofyear >= doy - 15) & (s.index.dayofyear <= doy + 15)] + baseline = seasonal if len(seasonal) >= 30 else s + pct = float((baseline < latest).mean() * 100) if len(baseline) else None + + lines.append( + f"**Record:** {len(s):,} daily values spanning **{years:.1f} years** " + f"({s.index.min():%Y-%m-%d} → {s.index.max():%Y-%m-%d})." + ) + if pct is not None: + lines.append( + f"**Latest flow:** **{latest:,.0f} cfs** on {latest_date:%Y-%m-%d} — " + f"about the **{pct:.0f}th percentile** for this time of year " + f"({_percentile_label(pct)})." + ) + lines.append("") + lines.append(summarize_selected_plots(plot_keys)) + lines.append("") + lines.append(summarize_flow_duration(df_q)) + lines.append("") + lines.append(summarize_seasonal_pattern(df_q)) + lines.append("") + lines.append(summarize_climate_merged(df_merged)) + rating = summarize_rating_curve(df_q) + if rating: + lines.append("") + lines.append(rating) + + # Computed metric strip + relevance + if not s.empty: + q10 = float(np.percentile(s, 90)) + q50 = float(np.percentile(s, 50)) + q90 = float(np.percentile(s, 10)) + mean = float(s.mean()) + peak = float(s.max()) + lines.append("") + lines.append("#### Key metrics (this period)") + lines.append( + f"- Mean **{mean:,.0f} cfs** · Median (Q50) **{q50:,.0f} cfs** · " + f"Q10 **{q10:,.0f} cfs** · Q90 **{q90:,.0f} cfs** · Peak day **{peak:,.0f} cfs**" + ) + if mean > q50 * 1.35: + lines.append( + "- **Mean ≫ median** — floods pull the average up; prefer median/Q50 for “typical” conditions." + ) + lines.append("") + lines.append( + format_metric_relevance_markdown( + metric_keys_for_plots(plot_keys), + df_q=df_q, + df_merged=df_merged, + plot_keys=plot_keys, + ) + ) + + lines.append("") + lines.append( + "_This is an automated screening narrative from the plotted data — not a formal " + "hydrologic design report. Verify peaks, ice, regulation, and rating shifts " + "before engineering use._" + ) + return "\n".join(lines) + + +def build_interactive_chart_brief( + df_q: pd.DataFrame | None, + df_merged: pd.DataFrame | None = None, +) -> str: + """Shorter narrative under auto-loaded interactive hydrograph + FDC.""" + s = _series(df_q) + if s.empty: + return "Interactive charts loaded without discharge values to summarize." + parts = [ + summarize_flow_duration(df_q), + summarize_seasonal_pattern(df_q), + "", + format_metric_relevance_markdown( + ["q10", "q50", "q90", "mean_flow"], + df_q=df_q, + df_merged=df_merged, + plot_keys=["flow_duration", "timeseries"], + ), + ] + return "\n\n".join(parts) + + +# Plain-English "why this metric matters" for dashboard cards and reports +METRIC_RELEVANCE: dict[str, dict[str, str]] = { + "record_length": { + "label": "Record length", + "meaning": "How many years of daily data are in the selected window.", + "relevance": ( + "Longer records support trends, frequency analysis, and climate normals. " + "Under ~10 years, treat flood frequency and long-term SPI carefully." + ), + "use_when": "Judging whether trends/frequency/SPI are defensible.", + }, + "data_points": { + "label": "Data points", + "meaning": "Count of daily observations after quality filtering.", + "relevance": ( + "More points reduce noise in percentiles and duration curves. " + "Large gaps matter more than total count for seasonal plots." + ), + "use_when": "Checking completeness before trusting summary stats.", + }, + "mean_flow": { + "label": "Mean flow", + "meaning": "Average daily discharge over the selected period.", + "relevance": ( + "Anchors water-supply style questions, but is pulled up by floods. " + "Compare to median (Q50) — if mean ≫ median, the record is peak-dominated." + ), + "use_when": "Rough water yield; always pair with median for skewed rivers.", + }, + "peak_flow": { + "label": "Peak flow", + "meaning": "Highest daily mean discharge in the selected period.", + "relevance": ( + "Flags flood magnitude in the window, not the official annual peak series. " + "Use Frequency Analysis (peak-flow table) for design return periods." + ), + "use_when": "Screening large events; not a substitute for LP3 frequency design.", + }, + "q10": { + "label": "Q10 (high flow)", + "meaning": "Discharge exceeded about 10% of days (upper duration curve).", + "relevance": "Describes wet-season / high-flow habitat and floodplain connectivity.", + "use_when": "High-flow ecology, channel maintenance, flood context.", + }, + "q50": { + "label": "Q50 (median)", + "meaning": "Discharge exceeded about half the days.", + "relevance": "Robust central tendency; less skewed by floods than the mean.", + "use_when": "Typical conditions and year-to-year comparisons.", + }, + "q90": { + "label": "Q90 (low flow)", + "meaning": "Discharge exceeded about 90% of days (lower duration curve).", + "relevance": "Drought, baseflow, and aquatic-habitat stress indicator.", + "use_when": "Low-flow management, summer shortages, baseflow screening.", + }, + "rating_r2": { + "label": "Rating R²", + "meaning": "How tightly stage and discharge follow the fitted Q–H model.", + "relevance": ( + "High R² means a stable control; scatter/seasonal color can flag hysteresis, " + "rating shifts, ice, or vegetation." + ), + "use_when": "Interpreting stage–discharge plots and gage reliability.", + }, + "spi_sri": { + "label": "SPI / SRI", + "meaning": "Standardized precip (SPI) or runoff (SRI) anomalies (γ → normal).", + "relevance": ( + "Negative = drier than normal for that accumulation window. " + "SRI lags SPI when soil/groundwater buffer the response." + ), + "use_when": "Drought monitoring and comparing meteorological vs hydrologic drought.", + }, +} + + +def metric_keys_for_plots(plot_keys: Iterable[str] | None = None) -> list[str]: + """Pick relevance keys that match the generated plot set.""" + keys = set(plot_keys) if plot_keys is not None else set() + out = ["record_length", "data_points", "mean_flow", "peak_flow", "q50", "q10", "q90"] + if any(k in keys for k in ("flow_duration", "low_flow_trend", "7q10_analysis")): + out.extend(["q10", "q90"]) + if "rating_curve" in keys: + out.append("rating_r2") + if any("spi" in k or "drought" in k or "precip" in k for k in keys): + out.append("spi_sri") + if not keys: + out = ["record_length", "data_points", "mean_flow", "peak_flow", "q10", "q50", "q90"] + seen: set[str] = set() + ordered: list[str] = [] + for k in out: + if k not in seen and k in METRIC_RELEVANCE: + seen.add(k) + ordered.append(k) + return ordered + + +def compute_hydrologic_profile( + df_q: pd.DataFrame | None, + df_merged: pd.DataFrame | None = None, + discharge_col: str | None = None, +) -> dict: + """Site/period stats that drive dynamic metric relevance.""" + s = _series(df_q, discharge_col) + profile: dict = {"ok": False} + if s.empty or len(s) < 5: + return profile + + q10 = float(np.percentile(s, 90)) + q50 = float(np.percentile(s, 50)) + q90 = float(np.percentile(s, 10)) + mean = float(s.mean()) + peak = float(s.max()) + years = max((s.index.max() - s.index.min()).days / 365.25, 0.01) + n = int(len(s)) + cv = float(s.std() / mean) if mean > 0 else float("nan") + ratio = q10 / max(q90, 1e-6) + skew = mean / max(q50, 1e-6) + + months = s.index.month + season_means: dict[str, float] = {} + for name, mlist in ( + ("DJF", [12, 1, 2]), + ("MAM", [3, 4, 5]), + ("JJA", [6, 7, 8]), + ("SON", [9, 10, 11]), + ): + sub = s[np.isin(months, mlist)] + if len(sub) >= 10: + season_means[name] = float(sub.mean()) + + wet_season = max(season_means, key=season_means.get) if season_means else None + dry_season = min(season_means, key=season_means.get) if season_means else None + season_factor = None + if wet_season and dry_season and season_means[dry_season] > 0: + season_factor = season_means[wet_season] / season_means[dry_season] + + if ratio >= 50 or (np.isfinite(cv) and cv >= 1.5): + regime, regime_plain = "flashy", "flashy / peak-dominated" + elif ratio >= 15 or (np.isfinite(cv) and cv >= 0.8): + regime, regime_plain = "seasonal", "seasonally variable" + else: + regime, regime_plain = "steady", "relatively steady / baseflow-supported" + + zero_frac = float((s <= max(q90 * 0.05, 0.01)).mean()) + + has_stage = ( + df_q is not None + and "Gage_Height_ft" in df_q.columns + and pd.to_numeric(df_q["Gage_Height_ft"], errors="coerce").notna().sum() >= 10 + ) + rating = None + if has_stage: + try: + from hydrology.analysis.stage_discharge import fit_best_rating_curve + + stage = pd.to_numeric(df_q["Gage_Height_ft"], errors="coerce") + q = pd.to_numeric(df_q["Discharge_cfs"], errors="coerce") + fit = fit_best_rating_curve(stage, q, min_points=10) + if fit.get("model") != "none" and np.isfinite(fit.get("R2", np.nan)): + rating = fit + except Exception: + rating = None + + climate: dict = {"has_precip": False, "has_temp": False} + if df_merged is not None and not df_merged.empty: + if "Precip_mm" in df_merged.columns: + p = pd.to_numeric(df_merged["Precip_mm"], errors="coerce").dropna() + if not p.empty: + climate["has_precip"] = True + if len(p) > 60: + climate["mean_annual_mm"] = float(p.resample("YE").sum().mean()) + else: + climate["mean_annual_mm"] = float(p.mean() * 365.25) + climate["wet_day_pct"] = float((p > 1.0).mean() * 100) + if "Temp_C" in df_merged.columns: + t = pd.to_numeric(df_merged["Temp_C"], errors="coerce").dropna() + if not t.empty: + climate["has_temp"] = True + climate["mean_temp_c"] = float(t.mean()) + + latest = float(s.iloc[-1]) + latest_date = s.index.max() + doy = latest_date.dayofyear + seasonal = s[(s.index.dayofyear >= doy - 15) & (s.index.dayofyear <= doy + 15)] + baseline = seasonal if len(seasonal) >= 30 else s + latest_pct = float((baseline < latest).mean() * 100) if len(baseline) else None + + profile.update( + { + "ok": True, + "n": n, + "years": years, + "q10": q10, + "q50": q50, + "q90": q90, + "mean": mean, + "peak": peak, + "cv": cv, + "q10_q90": ratio, + "mean_median": skew, + "regime": regime, + "regime_plain": regime_plain, + "season_means": season_means, + "wet_season": wet_season, + "dry_season": dry_season, + "season_factor": season_factor, + "zero_frac": zero_frac, + "has_stage": has_stage, + "rating": rating, + "climate": climate, + "latest": latest, + "latest_date": latest_date, + "latest_pct": latest_pct, + "start": s.index.min(), + "end": s.index.max(), + } + ) + return profile + + +def dynamic_metric_relevance( + df_q: pd.DataFrame | None, + df_merged: pd.DataFrame | None = None, + plot_keys: Iterable[str] | None = None, + discharge_col: str | None = None, + metric_keys: Iterable[str] | None = None, +) -> list[dict[str, str]]: + """Metric relevance rows computed from this gage/period's hydrology.""" + profile = compute_hydrologic_profile(df_q, df_merged, discharge_col) + if metric_keys is not None: + keys = [k for k in metric_keys if k in METRIC_RELEVANCE] + else: + keys = metric_keys_for_plots(plot_keys or []) + + if not profile.get("ok"): + return metric_relevance_table(keys) + + years, n = profile["years"], profile["n"] + q10, q50, q90 = profile["q10"], profile["q50"], profile["q90"] + mean, peak = profile["mean"], profile["peak"] + ratio, skew = profile["q10_q90"], profile["mean_median"] + regime = profile["regime_plain"] + wet, dry = profile.get("wet_season"), profile.get("dry_season") + sfactor = profile.get("season_factor") + + here = { + "record_length": ( + f"This selection covers **{years:.1f} years**. " + + ( + "Long enough for rough frequency/SPI screening." + if years >= 10 + else "Short for design frequency — treat extremes as exploratory only." + ) + ), + "data_points": ( + f"**{n:,}** daily values in window. " + + ( + "Dense enough for stable percentiles." + if n >= 365 * 3 + else "Thin sample — duration quantiles can jump if you change dates." + ) + ), + "mean_flow": ( + f"Mean **{mean:,.0f} cfs** vs median **{q50:,.0f} cfs** " + f"(mean is **{skew:.2f}×** the median). " + + ( + "Floods dominate the average — lean on Q50 for “typical” conditions." + if skew >= 1.35 + else "Mean and median are close — the series is not heavily peak-skewed." + ) + ), + "peak_flow": ( + f"Highest daily mean in this window is **{peak:,.0f} cfs** " + f"(**{peak / max(q50, 1e-6):.1f}×** the median). " + "Window peak only — not a formal annual-max design flood." + ), + "q10": ( + f"Q10 ≈ **{q10:,.0f} cfs** (exceeded ~10% of days). " + + ( + f"Q10/Q90 ≈ **{ratio:.0f}** → high flows sit far above baseflow ({regime})." + if ratio >= 15 + else f"Q10/Q90 ≈ {ratio:.1f} → high-flow contrast is mild for this period." + ) + ), + "q50": ( + f"Median **{q50:,.0f} cfs** is the best single “normal day” summary here. " + + ( + f"Wet season **{wet}** vs dry **{dry}** (~{sfactor:.1f}× contrast)." + if wet and dry and sfactor + else "Seasonal contrast is weak or not resolved in this window." + ) + ), + "q90": ( + f"Q90 ≈ **{q90:,.0f} cfs** (about 90% of days are at least this wet). " + + ( + f"Near-dry days are common (~{profile['zero_frac']*100:.0f}% near-zero)." + if profile["zero_frac"] >= 0.05 + else "Near-zero flow days are uncommon in this window." + ) + + f" Low-flow relevance is high for a **{regime}** river." + ), + "rating_r2": ( + ( + f"Best rating fit R²=**{profile['rating']['R2']:.3f}** " + f"(`{profile['rating']['equation']}`). " + + ( + "Offset H₀ correction active — gage zero ≠ zero-flow control." + if profile["rating"]["model"] == "offset_powerlaw" + else "Simple power-law is adequate for this pair set." + ) + ) + if profile.get("rating") + else ( + "Stage present but rating fit unavailable." + if profile.get("has_stage") + else "No gage-height in this window — rating metrics do not apply." + ) + ), + "spi_sri": ( + ( + "Climate is merged for this period" + + ( + f" (~{profile['climate'].get('mean_annual_mm', 0):.0f} mm/yr precip)" + if profile["climate"].get("has_precip") + else "" + ) + + f" — SPI/SRI should be read against this **{regime}** flow regime." + ) + if profile.get("climate", {}).get("has_precip") + or profile.get("climate", {}).get("has_temp") + else ( + "Climate is not merged into discharge for this period; " + "SPI may still run via Open-Meteo, but co-plotted climate metrics are limited." + ) + ), + } + + rows: list[dict[str, str]] = [] + for key in keys: + meta = METRIC_RELEVANCE.get(key) + if not meta: + continue + observed = { + "record_length": f"{years:.1f} yrs", + "data_points": f"{n:,}", + "mean_flow": f"{mean:,.0f} cfs", + "peak_flow": f"{peak:,.0f} cfs", + "q10": f"{q10:,.0f} cfs", + "q50": f"{q50:,.0f} cfs", + "q90": f"{q90:,.0f} cfs", + "rating_r2": ( + f"R²={profile['rating']['R2']:.3f}" if profile.get("rating") else "n/a" + ), + "spi_sri": ( + "climate linked" + if profile.get("climate", {}).get("has_precip") + else "climate limited" + ), + }.get(key, "—") + + rows.append( + { + "Metric": meta["label"], + "This site/period": observed, + "What it is": meta["meaning"], + "Why it matters here": here.get(key, meta["relevance"]), + "Use it when…": meta["use_when"], + "Regime": regime, + } + ) + return rows + + +def metric_relevance_table( + keys: Iterable[str] | None = None, +) -> list[dict[str, str]]: + """Static catalog rows (fallback when no discharge series).""" + use_keys = list(keys) if keys is not None else list(METRIC_RELEVANCE.keys()) + rows = [] + for key in use_keys: + meta = METRIC_RELEVANCE.get(key) + if not meta: + continue + rows.append( + { + "Metric": meta["label"], + "This site/period": "—", + "What it is": meta["meaning"], + "Why it matters here": meta["relevance"], + "Use it when…": meta["use_when"], + "Regime": "—", + } + ) + return rows + + +def format_metric_relevance_markdown( + keys: Iterable[str] | None = None, + df_q: pd.DataFrame | None = None, + df_merged: pd.DataFrame | None = None, + plot_keys: Iterable[str] | None = None, +) -> str: + """Markdown metric relevance — dynamic when discharge data is provided.""" + use_keys = list(keys) if keys is not None else None + if df_q is not None: + # Prefer explicit metric keys; else derive from plot set + if use_keys is not None and all(k in METRIC_RELEVANCE for k in use_keys): + rows = dynamic_metric_relevance( + df_q, df_merged, metric_keys=use_keys + ) + else: + rows = dynamic_metric_relevance( + df_q, df_merged, plot_keys=plot_keys or use_keys + ) + profile = compute_hydrologic_profile(df_q, df_merged) + else: + rows = metric_relevance_table(use_keys) + profile = {} + + if not rows: + return "" + + if profile.get("ok"): + header = ( + f"#### Metric relevance — **{profile['regime_plain']}** regime " + f"(Q10/Q90≈{profile['q10_q90']:.1f})" + ) + blurb = ( + "Each line uses **this site’s numbers** for the selected period " + "(not generic textbook text)." + ) + else: + header = "#### Metric relevance" + blurb = "What each number means and when to trust it:" + + lines = [header, blurb, ""] + for row in rows: + site_bit = row.get("This site/period") or "—" + lines.append( + f"- **{row['Metric']}** (**{site_bit}**) — {row['What it is']} " + f"*{row['Why it matters here']}*" + ) + return "\n".join(lines) + + +def metric_help_text( + key: str, + df_q: pd.DataFrame | None = None, + df_merged: pd.DataFrame | None = None, + discharge_col: str | None = None, +) -> str: + """Short tooltip for st.metric — site-specific when data is available.""" + meta = METRIC_RELEVANCE.get(key, {}) + base = " ".join( + p for p in (meta.get("meaning", ""), meta.get("use_when", "")) if p + ) + if df_q is None: + rel = meta.get("relevance", "") + return " ".join(p for p in (base, rel) if p) + + rows = dynamic_metric_relevance( + df_q, df_merged, discharge_col=discharge_col, metric_keys=[key] + ) + if not rows: + rel = meta.get("relevance", "") + return " ".join(p for p in (base, rel) if p) + + row = rows[0] + site = row.get("This site/period", "") + why = row.get("Why it matters here", "") + # Strip markdown for Streamlit tooltips + why_plain = why.replace("**", "").replace("`", "") + site_plain = site.replace("**", "").replace("`", "") + bits = [base] + if site_plain and site_plain != "—": + bits.append(f"This site/period: {site_plain}.") + if why_plain: + bits.append(why_plain) + return " ".join(bits) diff --git a/hydrology/app/page_modules/alerts.py b/hydrology/app/page_modules/alerts.py index 69842ac..eebcc7f 100644 --- a/hydrology/app/page_modules/alerts.py +++ b/hydrology/app/page_modules/alerts.py @@ -12,7 +12,7 @@ from hydrology.app.shared import ( get_inventory, get_cached_site_info, get_weather_station_info, - extract_site_id, display_site_info, site_picker, + extract_site_id, display_site_info, site_picker, fetch_climate_cached_result, logger) from hydrology.data.usgs import ( fetch_daily_values, fetch_instantaneous_values, @@ -38,9 +38,14 @@ def show(): st.error(f"Site {site_id} not found") return + render_site_current_check(site_id, site_info, key_prefix="alert_page") + + +def render_site_current_check(site_id: str, site_info: dict, key_prefix: str = "current_check"): + """Render a manual current threshold check for a selected site.""" desc = site_info.get('description', site_id) - st.header("Alert Check") + st.subheader("Current Conditions Check") st.caption( f"Manual threshold check for {desc}. Streamlit does not run background monitoring " "unless it is connected to a scheduled job or notification service." @@ -51,18 +56,18 @@ def show(): with col1: st.subheader("Flood Alerts") - flood_enabled = st.checkbox("Enable flood alerts", value=True) + flood_enabled = st.checkbox("Enable flood alerts", value=True, key=f"{key_prefix}_flood_enabled") if flood_enabled: - action_stage = st.number_input("Action Stage (ft)", value=10.0, step=0.5) - flood_stage = st.number_input("Flood Stage (ft)", value=12.0, step=0.5) - major_flood = st.number_input("Major Flood Stage (ft)", value=15.0, step=0.5) + action_stage = st.number_input("Action Stage (ft)", value=10.0, step=0.5, key=f"{key_prefix}_action_stage") + flood_stage = st.number_input("Flood Stage (ft)", value=12.0, step=0.5, key=f"{key_prefix}_flood_stage") + major_flood = st.number_input("Major Flood Stage (ft)", value=15.0, step=0.5, key=f"{key_prefix}_major_flood") with col2: st.subheader("Low Flow Alerts") - low_flow_enabled = st.checkbox("Enable low flow alerts", value=False) + low_flow_enabled = st.checkbox("Enable low flow alerts", value=False, key=f"{key_prefix}_low_flow_enabled") if low_flow_enabled: - low_flow_threshold = st.number_input("Low Flow (cfs)", value=100.0, step=10.0) - critical_flow = st.number_input("Critical Flow (cfs)", value=50.0, step=10.0) + low_flow_threshold = st.number_input("Low Flow (cfs)", value=100.0, step=10.0, key=f"{key_prefix}_low_flow") + critical_flow = st.number_input("Critical Flow (cfs)", value=50.0, step=10.0, key=f"{key_prefix}_critical_flow") st.markdown("---") @@ -72,7 +77,7 @@ def show(): icon="ℹ️", ) - if st.button("Run Current Check", type="primary"): + if st.button("Check Current Conditions", type="primary", key=f"{key_prefix}_run"): with st.spinner("Fetching current data..."): monitor = AlertMonitor() @@ -88,7 +93,7 @@ def show(): alerts = monitor.check_site(site_id, use_instantaneous=True) - st.subheader("Current Check") + st.subheader("Current Reading") try: end_date = datetime.now() @@ -228,14 +233,20 @@ def show(): lat = site_info.get('latitude') lon = site_info.get('longitude') if lat and lon: - from hydrology.data.climate import fetch_climate_data precip_start = (end_date - timedelta(days=7)).strftime('%Y-%m-%d') precip_end = end_date.strftime('%Y-%m-%d') - climate_df = fetch_climate_data(float(lat), float(lon), precip_start, precip_end) - - if climate_df is not None and 'prcp' in climate_df.columns: - total_precip_mm = climate_df['prcp'].sum() + climate_result = fetch_climate_cached_result( + float(lat), float(lon), + precip_start, precip_end, + site_id=site_id, + include_temp=False, + include_precip=True, + ) + climate_df = climate_result.get("data") + + if climate_df is not None and 'Precip_mm' in climate_df.columns: + total_precip_mm = climate_df['Precip_mm'].sum() total_precip_in = total_precip_mm / 25.4 if total_precip_in > 2: @@ -249,7 +260,7 @@ def show(): st.metric("7-Day Precipitation", f"{total_precip_in:.2f} in", delta=precip_status, delta_color="off", - help="Total precipitation in the last 7 days from nearest weather station") + help=f"Total precipitation in the last 7 days from {climate_result.get('source', 'the climate source')}") else: st.metric("7-Day Precipitation", "N/A", help="Total precipitation in the last 7 days from nearest weather station") diff --git a/hydrology/app/page_modules/comparisons.py b/hydrology/app/page_modules/comparisons.py index fabf9e5..ee7083c 100644 --- a/hydrology/app/page_modules/comparisons.py +++ b/hydrology/app/page_modules/comparisons.py @@ -15,6 +15,7 @@ create_comparison_figure, render_export_buttons, site_picker, logger) from hydrology.app.styles import render_site_header, render_plot_capability_board +from hydrology.app.page_modules.advanced import _multisite_analysis from hydrology.visualization.plots import AVAILABLE_PLOTS from hydrology.visualization.interactive import interactive_comparison, interactive_hydrograph @@ -34,9 +35,10 @@ def show(): st.header("Comparisons") - tab_time, tab_sites, tab_quad = st.tabs([ + tab_time, tab_sites, tab_relationships, tab_quad = st.tabs([ "Compare Time Periods", "Compare Sites", + "Site Relationships", "2x2 Comparison" ]) @@ -46,6 +48,9 @@ def show(): with tab_sites: _compare_sites(inventory_df) + with tab_relationships: + _multisite_analysis(inventory_df) + with tab_quad: _quad_comparison(inventory_df) diff --git a/hydrology/app/page_modules/indicators.py b/hydrology/app/page_modules/indicators.py index ca70310..68d7d13 100644 --- a/hydrology/app/page_modules/indicators.py +++ b/hydrology/app/page_modules/indicators.py @@ -14,13 +14,25 @@ from hydrology.app.shared import ( get_inventory, get_cached_site_info, - site_picker, logger) + site_picker, fetch_climate_cached_result, logger) from hydrology.app.interpretation import InsightCard, describe_standardized_index from hydrology.app.styles import render_insight_board from hydrology.data.usgs import fetch_daily_values, DEFAULT_PARAM_DISCHARGE from hydrology.analysis.indicators import ( calculate_spi, calculate_sri, classify_drought, calculate_baseflow_index_timeseries, get_seasonal_anomaly) +from hydrology.analysis.baseflow import compare_baseflow_methods + + +def _summarize_baseflow_methods(flow: pd.Series) -> dict: + """Return dashboard-ready baseflow method comparison labels.""" + comparison = compare_baseflow_methods(flow) + return { + "Lyne-Hollick BFI": f"{comparison['lyne_hollick_bfi']:.2f}", + "Eckhardt BFI": f"{comparison['eckhardt_bfi']:.2f}", + "Difference": f"{comparison['bfi_difference']:.2f}", + "Agreement": str(comparison["agreement"]).title(), + } def show(): @@ -41,6 +53,11 @@ def show(): st.error(f"Site {site_id} not found") return + render_site_indicators(site_id, site_info) + + +def render_site_indicators(site_id, site_info): + """Render standardized runoff, precipitation, baseflow, and seasonal indicators for one site.""" desc = site_info.get('description', site_id) lat = site_info.get('latitude') lon = site_info.get('longitude') @@ -105,24 +122,70 @@ def _render_drought_tab(df_q, q_col, site_id, desc, lat, lon, start_str, end_str # SPI if climate data available st.markdown("---") st.subheader("Standardized Precipitation Index (SPI)") - show_spi = st.checkbox("Calculate SPI (requires climate data)", value=False, key="show_spi") + st.caption( + "Precipitation is resolved from the nearest Meteostat station (Daymet only if " + "NASA Earthdata credentials are configured). SPI fits need ~10+ years of monthly " + "precip — climate is auto-extended to at least 30 years when possible." + ) + # Climate window: SPI-12 needs long monthly history (independent of discharge slider) + spi_start = (datetime.strptime(end_str, "%Y-%m-%d") - timedelta(days=30 * 365)).strftime( + "%Y-%m-%d" + ) + spi_key = f"spi_result_{site_id}_{spi_start}_{end_str}" + calculate_spi_clicked = st.button( + "Calculate SPI from precipitation", + key=f"calculate_spi_{site_id}", + type="primary", + ) - if show_spi: - with st.spinner("Fetching climate data..."): - precip_data = _fetch_precip_data(site_id, lat, lon, start_str, end_str) + if calculate_spi_clicked: + with st.spinner("Fetching precipitation (Meteostat / Daymet)..."): + precip_result = _fetch_precip_data_result( + site_id, lat, lon, spi_start, end_str + ) + precip_data = precip_result["precip"] if precip_data is not None and not precip_data.empty: spi_df = calculate_spi(precip_data, windows=[1, 3, 6, 12]) + n_months = int(precip_data.resample("ME").sum().shape[0]) if len(precip_data) else 0 + precip_result = { + **precip_result, + "n_months": n_months, + "climate_start": spi_start, + "climate_end": end_str, + } if not spi_df.empty: - _render_index_interpretation(spi_df, "SPI", "precipitation") - fig_spi = _create_drought_timeseries(spi_df, title=f"{desc} - SPI") - st.plotly_chart(fig_spi, width="stretch", key="spi_chart") + st.session_state[spi_key] = {"fetch": precip_result, "spi": spi_df} else: - st.warning("Insufficient precipitation data for SPI. Try a longer period or a site with stronger climate coverage.") + precip_result["message"] = ( + precip_result.get("message", "") + + f" Precipitation loaded ({n_months} months) but SPI fit failed — " + "need enough complete months per calendar month (≈10+ years)." + ).strip() + st.session_state[spi_key] = {"fetch": precip_result, "spi": pd.DataFrame()} else: + st.session_state[spi_key] = {"fetch": precip_result, "spi": pd.DataFrame()} + + spi_result = st.session_state.get(spi_key) + if spi_result: + st.dataframe( + pd.DataFrame(_spi_readiness_rows(spi_result["fetch"])), + width="stretch", + hide_index=True, + ) + spi_df = spi_result["spi"] + if not spi_df.empty: + _render_index_interpretation(spi_df, "SPI", "precipitation") + fig_spi = _create_drought_timeseries(spi_df, title=f"{desc} - SPI") + st.plotly_chart(fig_spi, width="stretch", key=f"spi_chart_{site_id}") + elif spi_result["fetch"].get("precip") is not None: st.warning( - "Could not fetch precipitation data. SPI uses Daymet first and then the nearest Meteostat station from the selected site coordinates." + "Precipitation was found but SPI could not be fit. " + "Use a longer period (SPI needs ~10+ years of monthly totals) " + "or try another site if the weather station coverage is sparse." ) + else: + st.warning(spi_result["fetch"].get("message") or "Precipitation unavailable.") def _render_drought_status_cards(sri_df: pd.DataFrame): @@ -241,7 +304,7 @@ def _render_bfi_tab(df_q, q_col, desc): st.subheader("Baseflow Index (BFI) Trend") st.caption( "Ratio of baseflow to total flow. Declining BFI may indicate " - "reduced groundwater contribution or aquifer depletion." + "reduced baseflow contribution; interpret as a groundwater proxy, not a direct aquifer measurement." ) window = st.slider("Rolling window (days)", 30, 365, 90, step=30, key="bfi_window") @@ -259,11 +322,11 @@ def _render_bfi_tab(df_q, q_col, desc): with col1: if current_bfi is not None: st.metric("Current BFI", f"{current_bfi:.2f}", - help="Baseflow Index: fraction of flow from groundwater vs. surface runoff. 0-1, higher = more groundwater") + help="Baseflow Index: fraction of flow separated as baseflow. 0-1, higher = more sustained/baseflow-dominated flow.") with col2: mean_bfi = bfi_df['bfi'].mean() st.metric("Mean BFI", f"{mean_bfi:.2f}", - help="Long-term average Baseflow Index. Higher = greater groundwater contribution") + help="Long-term average Baseflow Index. Higher = more sustained/baseflow-dominated flow.") with col3: if current_bfi is not None: delta = current_bfi - mean_bfi @@ -271,6 +334,13 @@ def _render_bfi_tab(df_q, q_col, desc): delta="Above" if delta > 0 else "Below", delta_color="normal" if delta > 0 else "inverse") + method_summary = _summarize_baseflow_methods(df_q[q_col]) + st.caption("Method comparison for the selected period") + method_cols = st.columns(4) + for col, (label, value) in zip(method_cols, method_summary.items()): + with col: + st.metric(label, value, delta_color="off") + # BFI time series fig = make_subplots( rows=2, cols=1, row_heights=[0.6, 0.4], @@ -406,32 +476,100 @@ def _render_anomaly_tab(df_q, q_col, desc): def _fetch_precip_data(site_id, lat, lon, start_str, end_str): """Try Daymet first, fall back to Meteostat for precipitation.""" + return _fetch_precip_data_result(site_id, lat, lon, start_str, end_str)["precip"] + + +def _valid_coords(lat, lon) -> tuple[float, float] | tuple[None, None]: + """Parse and validate lat/lon (reject None/NaN/out-of-range).""" try: - from hydrology.data.hyriver import get_daymet_climate - daymet = get_daymet_climate(site_id, start_str, end_str, variables=['prcp']) - if daymet is not None and 'precip_mm' in daymet.columns: - return daymet['precip_mm'] - except ImportError: - pass - except Exception as e: - logger.debug(f"Daymet failed, trying Meteostat: {e}") + lat_f = float(lat) + lon_f = float(lon) + except (TypeError, ValueError): + return None, None + if not (np.isfinite(lat_f) and np.isfinite(lon_f)): + return None, None + if not (-90.0 <= lat_f <= 90.0 and -180.0 <= lon_f <= 180.0): + return None, None + return lat_f, lon_f + + +def _fetch_precip_data_result(site_id, lat, lon, start_str, end_str): + """Fetch precipitation and return source metadata for the dashboard.""" + lat_f, lon_f = _valid_coords(lat, lon) + if lat_f is None or lon_f is None: + return { + "precip": None, + "source": "Unavailable", + "n_days": 0, + "message": ( + "Could not fetch precipitation because the selected site is missing " + "valid latitude/longitude in the inventory." + ), + } - # Meteostat fallback try: - from hydrology.data.climate import fetch_climate_data - if lat and lon: - climate = fetch_climate_data( - float(lat), float(lon), - pd.Timestamp(start_str), pd.Timestamp(end_str), - include_temp=False, include_precip=True) - if climate is not None: - if 'Precip_mm' in climate.columns: - return climate['Precip_mm'] - if 'precip_mm' in climate.columns: - return climate['precip_mm'] - if 'prcp' in climate.columns: - return climate['prcp'] + result = fetch_climate_cached_result( + lat_f, + lon_f, + start_str, + end_str, + site_id=site_id, + include_temp=False, + include_precip=True, + ) + climate = result.get("data") + if climate is not None and "Precip_mm" in climate.columns: + precip = climate["Precip_mm"].dropna() + if not precip.empty: + return { + "precip": precip, + "source": result.get("source", "Climate"), + "n_days": len(precip), + "message": result.get("message", "Loaded precipitation data."), + "station_name": result.get("station_name"), + "station_distance_km": result.get("station_distance_km"), + } + # Pass through provider message when data missing + return { + "precip": None, + "source": result.get("source", "Unavailable"), + "n_days": 0, + "message": result.get( + "message", + "Could not fetch precipitation from Meteostat or Daymet for this site and date range.", + ), + "station_name": result.get("station_name"), + "station_distance_km": result.get("station_distance_km"), + } except Exception as e: - logger.debug(f"Meteostat precip fetch failed: {e}") - - return None + logger.debug(f"Precipitation fetch failed: {e}") + return { + "precip": None, + "source": "Unavailable", + "n_days": 0, + "message": f"Precipitation fetch error: {e}", + } + + +def _spi_readiness_rows(fetch_result): + """Build compact source/status rows for SPI calculation.""" + dist = fetch_result.get("station_distance_km") + dist_txt = f"{float(dist):.1f} km" if dist is not None else "—" + station = fetch_result.get("station_name") or "—" + climate_range = "—" + if fetch_result.get("climate_start") and fetch_result.get("climate_end"): + climate_range = f"{fetch_result['climate_start']} → {fetch_result['climate_end']}" + return [ + {"Item": "Precipitation source", "Value": fetch_result.get("source", "Unavailable")}, + {"Item": "Weather station", "Value": station}, + {"Item": "Distance to gage", "Value": dist_txt}, + {"Item": "Climate window", "Value": climate_range}, + {"Item": "Daily precipitation records", "Value": f"{int(fetch_result.get('n_days', 0)):,}"}, + { + "Item": "Monthly totals", + "Value": f"{int(fetch_result.get('n_months', 0)):,}" + if fetch_result.get("n_months") is not None + else "—", + }, + {"Item": "Status", "Value": fetch_result.get("message", "")}, + ] diff --git a/hydrology/app/page_modules/overview.py b/hydrology/app/page_modules/overview.py index 45d7278..f19188f 100644 --- a/hydrology/app/page_modules/overview.py +++ b/hydrology/app/page_modules/overview.py @@ -24,6 +24,7 @@ render_status_chips ) from hydrology.app.interpretation import summarize_flow_context +from hydrology.app.page_modules.alerts import render_site_current_check from hydrology.data.usgs import ( fetch_daily_values, fetch_instantaneous_values, DEFAULT_PARAM_DISCHARGE, DEFAULT_PARAM_STAGE @@ -66,24 +67,23 @@ def _mini_sparkline(series, height=50): def _render_regional_summary(): """Show a compact conditions table for local/priority sites.""" - from hydrology.data.usgs import fetch_current_conditions, fetch_daily_percentiles, classify_condition from hydrology.visualization.map_utils import get_condition_color, get_condition_label site_ids = list(LOCAL_SITES.keys()) - current = fetch_current_conditions(site_ids) - percentiles = fetch_daily_percentiles(site_ids) + # Use shared cache (ttl=3600) — avoids double-fetching on every Stations rerun + details = get_site_condition_details(site_ids) rows = [] for sid, name in LOCAL_SITES.items(): - flow = current.get(sid) - pcts = percentiles.get(sid) - pctile = classify_condition(flow, pcts) if flow and pcts else None + info = details.get(sid) or {} + flow = info.get("flow_cfs") + pctile = info.get("percentile") label = get_condition_label(pctile) if pctile is not None else "N/A" color = get_condition_color(pctile) if pctile is not None else "#808080" rows.append({ "Site": name, - "Flow (cfs)": f"{flow:,.0f}" if flow else "N/A", + "Flow (cfs)": f"{flow:,.0f}" if flow is not None else "N/A", "Condition": label, "_color": color, "_site_id": sid, @@ -92,13 +92,18 @@ def _render_regional_summary(): if not rows: return - # Render as styled cards in columns + # Regional current conditions — wrapped for consistent card polish/animation + render_workspace_panel( + "Regional Current Conditions", + "Live flow and USGS seasonal percentile context for priority PNW sites.", + [{"label": "Cached 1h", "state": "ready"}, {"label": f"{len(rows)} gages", "state": "ready"}], + ) cols = st.columns(len(rows)) for col, row in zip(cols, rows): with col: st.markdown( - f'<div style="border-left: 3px solid {row["_color"]}; ' - f'padding-left: 0.5rem; margin-bottom: 0.5rem;">' + f'<div class="status-chip-row" style="border-left: 3px solid {row["_color"]}; ' + f'padding-left: 0.5rem; margin-bottom: 0.35rem;">' f'<div style="font-size: 0.75rem; color: #8899a6;">{row["Site"]}</div>' f'<div style="font-size: 1.1rem; font-weight: 600; color: #e0e0e0;">{row["Flow (cfs)"]}</div>' f'<div style="font-size: 0.7rem; color: {row["_color"]};">{row["Condition"]}</div>' @@ -142,6 +147,10 @@ def show(): st.caption("Fast context from the last 10 years of daily discharge.") render_insight_board(summarize_flow_context(df_hist)) + with st.expander("Current Conditions Check", expanded=False): + st.caption("Run threshold checks for the selected gage without leaving Stations.") + render_site_current_check(site_id, site_info, key_prefix=f"overview_current_{site_id}") + _render_quick_stats(df_hist) @@ -163,11 +172,6 @@ def _render_site_workspace(site_id: str, site_info: dict, condition: dict | None "href": f"comparisons?site={site_id}", "body": "Check overlap against nearby or selected gages.", }, - { - "title": "Current Check", - "href": f"alerts?site={site_id}", - "body": "Run manual threshold checks for this gage.", - }, ]) diff --git a/hydrology/app/page_modules/reach_analysis.py b/hydrology/app/page_modules/reach_analysis.py index 6a1e26f..035cef5 100644 --- a/hydrology/app/page_modules/reach_analysis.py +++ b/hydrology/app/page_modules/reach_analysis.py @@ -6,6 +6,7 @@ import streamlit as st import matplotlib.pyplot as plt from datetime import date +import json from hydrology.app.shared import ( get_inventory, get_cached_site_info, @@ -13,21 +14,713 @@ date_range_selector, render_export_buttons, _filter_inventory, FIPS_TO_STATE, logger) -from hydrology.app.styles import render_site_header from hydrology.app.plot_config import REACH_PLOTS, get_display_name from hydrology.visualization.plots import AVAILABLE_PLOTS from hydrology.core import DEFAULT_DISCHARGE_CODE -from hydrology.core.timezone import ensure_utc -from hydrology.data.climate import fetch_climate_data from hydrology.app.shared import fetch_climate_cached from hydrology.visualization.interactive import baseflow_waterfall +from hydrology.analysis.reach_gain_loss import summarize_reach_gain_loss +from hydrology.data.nldi import discover_related_sites +from hydrology.app.styles import render_workspace_panel, render_insight_board import pandas as pd -# Default Spokane River reach stations -DEFAULT_UPSTREAM = "12419000" # Post Falls -DEFAULT_DOWNSTREAM = "12422500" # Spokane at Spokane (Greene St) +def _build_reach_summary_row(upstream_id, downstream_id, upstream_q, downstream_q, reach_km=None): + """Build one dashboard row for paired reach gain/loss.""" + summary = summarize_reach_gain_loss(upstream_q, downstream_q, reach_km=reach_km) + median_gain = summary.get("median_gain_cfs") + low_flow_gain = summary.get("low_flow_median_gain_cfs") + per_km = summary.get("median_gain_cfs_per_km") + return { + "Reach": f"{upstream_id} -> {downstream_id}", + "Class": summary.get("classification", "insufficient_data"), + "Median gain/loss": f"{median_gain:,.0f} cfs" if pd.notna(median_gain) else "N/A", + "Low-flow gain/loss": f"{low_flow_gain:,.0f} cfs" if pd.notna(low_flow_gain) else "N/A", + "Gain/loss per km": f"{per_km:,.1f} cfs/km" if pd.notna(per_km) else "Add reach length", + "Confidence": summary.get("confidence", "none"), + } + + +def _build_reach_interpretation(reach_row, reach_km=None, length_source="missing"): + """Convert reach screening metrics into a concise dashboard interpretation.""" + reach_class = str(reach_row.get("Class", "insufficient_data")).lower() + confidence = str(reach_row.get("Confidence", "none")).lower() + per_km = reach_row.get("Gain/loss per km", "Add reach length") + + if reach_class == "gaining": + finding = "Gaining reach" + direction_text = "downstream flow is higher than upstream flow" + elif reach_class == "losing": + finding = "Losing reach" + direction_text = "downstream flow is lower than upstream flow" + elif reach_class == "neutral": + finding = "No clear gain/loss" + direction_text = "upstream and downstream flow are similar over the paired record" + else: + finding = "Not enough paired data" + direction_text = "paired daily flow overlap is not sufficient for a reach call" + + if reach_km: + length_label = f"{reach_km:.1f} km, {length_source} inferred" if length_source == "network" else f"{reach_km:.1f} km, manual" + length_text = f"Normalized gain/loss is {per_km}." + else: + length_label = "Not inferred" + length_text = "cfs/km is unavailable until the network length is inferred or entered." + + review = "Use as a screening result." + if confidence != "high": + review = "Screening result; check tributaries, diversions, withdrawals, and data overlap." + + return { + "Finding": finding, + "Confidence": confidence, + "Median gain/loss": reach_row.get("Median gain/loss", "N/A"), + "Low-flow gain/loss": reach_row.get("Low-flow gain/loss", "N/A"), + "Reach length": length_label, + "Interpretation": f"{direction_text}; {length_text}", + "Review": review, + } + + +def _format_reach_chain(site_ids): + """Format adjacent reaches in selected upstream-to-downstream order.""" + return [ + {"Order": idx, "Reach": f"{upstream} -> {downstream}"} + for idx, (upstream, downstream) in enumerate(zip(site_ids, site_ids[1:]), start=1) + ] + + +def _signed_navigation_distance(site_id, related_sites, origin_site_id): + """Return signed distance from an NLDI anchor; upstream is negative.""" + if site_id == origin_site_id: + return 0.0 + for site in related_sites: + if str(site.get("site_id")) != str(site_id): + continue + distance = site.get("distance_km") + direction = str(site.get("direction", "")).lower() + if distance is None or direction not in {"upstream", "downstream"}: + return None + signed = float(distance) + return -signed if direction == "upstream" else signed + return None + + +def _estimate_reach_km(upstream_id, downstream_id, related_sites, origin_site_id): + """Estimate reach length from navigation distances relative to one anchor.""" + upstream_distance = _signed_navigation_distance(upstream_id, related_sites, origin_site_id) + downstream_distance = _signed_navigation_distance(downstream_id, related_sites, origin_site_id) + if upstream_distance is None or downstream_distance is None: + return None + reach_km = abs(downstream_distance - upstream_distance) + return round(reach_km, 2) if reach_km > 0 else None + + +def _resolve_reach_km(estimated_reach_km, manual_reach_km): + """Use inferred network length unless only a manual override is available.""" + if estimated_reach_km and estimated_reach_km > 0: + return estimated_reach_km + if manual_reach_km and manual_reach_km > 0: + return manual_reach_km + return None + + +def _flowline_distance_km(reach_km, search_km): + """Choose a bounded flowline lookup distance for reach mapping.""" + if reach_km and reach_km > 0: + return min(max(round(reach_km * 1.5, 1), 10.0), 300.0) + return float(search_km) + + +def _flowline_style(selected=False): + """Folium style for reach network flowlines.""" + if selected: + return { + "color": "#ffd166", + "weight": 6, + "opacity": 0.95, + } + return { + "color": "#4fc3f7", + "weight": 2, + "opacity": 0.45, + } + + +def _clip_flowlines_between_gages(flowlines, upstream_lat, upstream_lon, downstream_lat, downstream_lon): + """Clip a flowline layer to the portion between selected upstream/downstream gages.""" + if flowlines is None or getattr(flowlines, "empty", True): + return None + + try: + import geopandas as gpd + from shapely.geometry import Point + + original_crs = flowlines.crs + work = flowlines + upstream_point = Point(float(upstream_lon), float(upstream_lat)) + downstream_point = Point(float(downstream_lon), float(downstream_lat)) + + if original_crs is not None and getattr(original_crs, "is_geographic", False): + work = flowlines.to_crs(epsg=3857) + points = gpd.GeoSeries([upstream_point, downstream_point], crs=original_crs).to_crs(epsg=3857) + upstream_point = points.iloc[0] + downstream_point = points.iloc[1] + + path = _trace_flowline_path_between_points(work.geometry, upstream_point, downstream_point) + if path is None or path.is_empty: + return None + + clipped_gdf = gpd.GeoDataFrame({"segment": ["selected reach"]}, geometry=[path], crs=work.crs) + if original_crs is not None and clipped_gdf.crs != original_crs: + clipped_gdf = clipped_gdf.to_crs(original_crs) + return clipped_gdf + except Exception as e: + logger.warning(f"Could not clip selected reach flowlines: {e}") + return None + + +def _iter_lines(geometries): + """Yield LineString parts from a geometry collection.""" + for geom in geometries: + if geom is None or geom.is_empty: + continue + if geom.geom_type == "LineString": + yield geom + elif geom.geom_type == "MultiLineString": + yield from geom.geoms + + +def _coord_key(coord, precision=6): + """Round map coordinates so shared NHD segment endpoints connect.""" + return (round(float(coord[0]), precision), round(float(coord[1]), precision)) + + +def _nearest_segment_projection(lines, point): + """Return the nearest segment plus the projected point on that segment.""" + from shapely.geometry import LineString + + best = None + for line_index, line in enumerate(lines): + coords = list(line.coords) + for segment_index, (start, end) in enumerate(zip(coords, coords[1:])): + segment = LineString([start, end]) + if segment.length == 0: + continue + distance_on_segment = segment.project(point) + projected = segment.interpolate(distance_on_segment) + distance_to_point = projected.distance(point) + candidate = (distance_to_point, line_index, segment_index, projected) + if best is None or candidate[0] < best[0]: + best = candidate + return best + + +def _add_graph_edge(graph, left_key, right_key, left_coord, right_coord): + """Add an undirected weighted edge to a coordinate graph.""" + from math import hypot + + if left_key == right_key: + return + weight = hypot(right_coord[0] - left_coord[0], right_coord[1] - left_coord[1]) + graph.setdefault(left_key, {})[right_key] = weight + graph.setdefault(right_key, {})[left_key] = weight + + +def _shortest_coord_path(graph, start_key, end_key): + """Dijkstra path through the flowline coordinate graph.""" + import heapq + + queue = [(0.0, start_key, [start_key])] + visited = set() + while queue: + distance, node, path = heapq.heappop(queue) + if node == end_key: + return path + if node in visited: + continue + visited.add(node) + for neighbor, weight in graph.get(node, {}).items(): + if neighbor not in visited: + heapq.heappush(queue, (distance + weight, neighbor, path + [neighbor])) + return None + + +def _trace_flowline_path_between_points(geometries, upstream_point, downstream_point): + """Trace the connected NHD path between two projected gage points.""" + from shapely.geometry import LineString, Point + + lines = list(_iter_lines(geometries)) + if not lines: + return None + + upstream_projection = _nearest_segment_projection(lines, upstream_point) + downstream_projection = _nearest_segment_projection(lines, downstream_point) + if upstream_projection is None or downstream_projection is None: + return None + + projection_by_segment = {} + _, up_line_idx, up_segment_idx, up_projected = upstream_projection + _, dn_line_idx, dn_segment_idx, dn_projected = downstream_projection + projection_by_segment.setdefault((up_line_idx, up_segment_idx), []).append(("upstream", up_projected)) + projection_by_segment.setdefault((dn_line_idx, dn_segment_idx), []).append(("downstream", dn_projected)) + + graph = {} + coordinates = {} + start_key = None + end_key = None + + for line_index, line in enumerate(lines): + coords = list(line.coords) + for segment_index, (segment_start, segment_end) in enumerate(zip(coords, coords[1:])): + segment = LineString([segment_start, segment_end]) + split_points = [(0.0, None, Point(segment_start))] + for label, projected in projection_by_segment.get((line_index, segment_index), []): + split_points.append((segment.project(projected), label, projected)) + split_points.append((segment.length, None, Point(segment_end))) + split_points = sorted(split_points, key=lambda item: item[0]) + + keyed_points = [] + for _, label, point in split_points: + key = _coord_key(point.coords[0]) + coordinates[key] = point.coords[0] + if label == "upstream": + start_key = key + elif label == "downstream": + end_key = key + keyed_points.append((key, point.coords[0])) + + for (left_key, left_coord), (right_key, right_coord) in zip(keyed_points, keyed_points[1:]): + _add_graph_edge(graph, left_key, right_key, left_coord, right_coord) + + if start_key is None or end_key is None: + return None + + path_keys = _shortest_coord_path(graph, start_key, end_key) + if not path_keys or len(path_keys) < 2: + return None + + return LineString([coordinates[key] for key in path_keys]) + + +def _map_bounds_for_reach( + selected_flowlines, + context_flowlines, + upstream_lat, + upstream_lon, + downstream_lat, + downstream_lon, +): + """Return folium bounds focused on the selected gages.""" + lat_min = min(float(upstream_lat), float(downstream_lat)) + lat_max = max(float(upstream_lat), float(downstream_lat)) + lon_min = min(float(upstream_lon), float(downstream_lon)) + lon_max = max(float(upstream_lon), float(downstream_lon)) + + lat_pad = max((lat_max - lat_min) * 0.25, 0.01) + lon_pad = max((lon_max - lon_min) * 0.25, 0.01) + return [ + [round(lat_min - lat_pad, 6), round(lon_min - lon_pad, 6)], + [round(lat_max + lat_pad, 6), round(lon_max + lon_pad, 6)], + ] + + +def _reach_map_component_key(upstream_id, downstream_id, bounds): + """Return a stable key that remounts the map when the selected reach changes.""" + flat_bounds = "_".join(f"{coord:.4f}" for row in bounds for coord in row) + return f"reach_map_{upstream_id}_{downstream_id}_{flat_bounds}" + + +def _leaflet_fit_bounds_script(bounds): + """Return a Folium-compatible script that forces Leaflet to fit selected reach bounds.""" + bounds_json = json.dumps(bounds) + return ( + "<script>" + "setTimeout(function(){" + "for (const key in window) {" + "const value = window[key];" + "if (value && value.fitBounds && value.eachLayer) {" + f"value.fitBounds({bounds_json}, {{paddingTopLeft:[24,24], paddingBottomRight:[24,24], maxZoom:13}});" + "}" + "}" + "}, 250);" + "</script>" + ) + + +def _filter_related_sites_to_inventory(origin_site_id, related_sites, inventory_df): + """Keep NLDI candidates that HydroPlot can resolve from its inventory.""" + if inventory_df is None or inventory_df.empty or "site_id" not in inventory_df.columns: + return [], [str(site.get("site_id")) for site in related_sites if site.get("site_id")] + + inventory_ids = set(inventory_df["site_id"].astype(str)) + filtered = [] + omitted = [] + for site in related_sites: + site_id = str(site.get("site_id", "")) + if not site_id: + continue + if site_id == str(origin_site_id) or site_id in inventory_ids: + filtered.append(site) + else: + omitted.append(site_id) + return filtered, omitted + + +def _format_related_site_rows(origin_site_id, related_sites): + """Make NLDI candidate stations readable for the reach-selection UI.""" + rows = [ + { + "Station": str(origin_site_id), + "Position": "Anchor", + "Distance from anchor": "0.0 km", + "Name": "Selected anchor station", + } + ] + for site in related_sites: + direction = str(site.get("direction", "unknown")).lower() + navigation_mode = str(site.get("navigation_mode", "")).lower() + if "trib" in navigation_mode: + position = "Tributary" + elif direction == "upstream": + position = "Upstream" + elif direction == "downstream": + position = "Downstream" + else: + position = "Unknown" + distance = site.get("distance_km") + rows.append( + { + "Station": str(site.get("site_id", "")), + "Position": position, + "Distance from anchor": f"{float(distance):.1f} km" if distance is not None else "Unknown", + "Name": site.get("name", ""), + } + ) + return rows + + +def _candidate_position(site): + """Return the user-facing network position for a candidate site.""" + direction = str(site.get("direction", "unknown")).lower() + navigation_mode = str(site.get("navigation_mode", "")).lower() + if "trib" in navigation_mode: + return "Tributary" + if direction == "upstream": + return "Upstream" + if direction == "downstream": + return "Downstream" + return "Anchor" + + +def _build_reach_candidate_options(origin_site_id, origin_name, related_sites): + """Build labeled gage options for reach selectors.""" + candidates = [ + { + "site_id": str(origin_site_id), + "label": f"Anchor | {origin_site_id} | 0.0 km | {origin_name}", + "position": "Anchor", + "distance_km": 0.0, + } + ] + for site in related_sites: + site_id = str(site.get("site_id", "")) + if not site_id: + continue + position = _candidate_position(site) + distance = site.get("distance_km") + distance_label = f"{float(distance):.1f} km" if distance is not None else "distance unknown" + name = site.get("name") or site.get("description") or "" + candidates.append( + { + "site_id": site_id, + "label": f"{position} | {site_id} | {distance_label} | {name}", + "position": position, + "distance_km": float(distance) if distance is not None else None, + } + ) + + position_order = {"Anchor": 0, "Upstream": 1, "Tributary": 2, "Downstream": 3} + candidates.sort( + key=lambda candidate: ( + position_order.get(candidate["position"], 9), + candidate["distance_km"] if candidate["distance_km"] is not None else 9999.0, + ) + ) + return candidates + + +def _candidate_display_name(candidate): + """Return the candidate station name without repeating selector metadata.""" + if candidate.get("name"): + return str(candidate["name"]) + label = str(candidate.get("label", "")) + parts = [part.strip() for part in label.split("|")] + return parts[-1] if len(parts) >= 4 else "" + + +def _format_pair_distance(distance_km): + """Format pair distance for compact candidate labels.""" + return f"{float(distance_km):.1f} km" if distance_km is not None else "distance unknown" + + +def _pair_key(upstream_id, downstream_id): + """Return a stable selected-reach key.""" + return f"{upstream_id}__{downstream_id}" + + +def _pair_label_for_key(reach_pairs, pair_key): + """Return a pair label for a selected reach key.""" + for pair in reach_pairs: + if pair.get("key") == pair_key: + return pair.get("label") + return None + + +def _cycle_pair_key(reach_pairs, current_key, step): + """Return the previous or next reach pair key, wrapping around the list.""" + if not reach_pairs: + return None + keys = [pair["key"] for pair in reach_pairs] + if current_key not in keys: + return keys[0] + current_index = keys.index(current_key) + return keys[(current_index + step) % len(keys)] + + +def _resolve_selected_pair_key(reach_pairs, session_state): + """Keep a selected reach pair if it remains valid; otherwise choose the first available pair.""" + if not reach_pairs: + return None + valid_keys = {pair["key"] for pair in reach_pairs} + current = session_state.get("reach_selected_pair_key") + if current in valid_keys: + return current + return reach_pairs[0]["key"] + + +def _build_recommended_reach_pairs(origin_site_id, candidates, max_pairs=12): + """Build processable upstream/downstream reach pairs for the workspace.""" + origin_site_id = str(origin_site_id) + by_position = {"Upstream": [], "Downstream": [], "Tributary": []} + for candidate in candidates: + site_id = str(candidate.get("site_id", "")) + if not site_id or site_id == origin_site_id: + continue + position = candidate.get("position") + if position in by_position: + by_position[position].append(candidate) + + def distance_value(candidate): + distance = candidate.get("distance_km") + return float(distance) if distance is not None else 9999.0 + + for values in by_position.values(): + values.sort(key=distance_value) + + pairs = [] + for upstream in by_position["Upstream"]: + distance_label = _format_pair_distance(upstream.get("distance_km")) + name = _candidate_display_name(upstream) + pairs.append({ + "key": _pair_key(upstream["site_id"], origin_site_id), + "upstream_id": str(upstream["site_id"]), + "downstream_id": origin_site_id, + "label": f'Upstream: {upstream["site_id"]} -> {origin_site_id} | {distance_label} | {name}', + "kind": "mainstem upstream", + "distance_km": upstream.get("distance_km"), + "name": name, + }) + for downstream in by_position["Downstream"]: + distance_label = _format_pair_distance(downstream.get("distance_km")) + name = _candidate_display_name(downstream) + pairs.append({ + "key": _pair_key(origin_site_id, downstream["site_id"]), + "upstream_id": origin_site_id, + "downstream_id": str(downstream["site_id"]), + "label": f'Downstream: {origin_site_id} -> {downstream["site_id"]} | {distance_label} | {name}', + "kind": "mainstem downstream", + "distance_km": downstream.get("distance_km"), + "name": name, + }) + for tributary in by_position["Tributary"]: + distance_label = _format_pair_distance(tributary.get("distance_km")) + name = _candidate_display_name(tributary) + pairs.append({ + "key": _pair_key(tributary["site_id"], origin_site_id), + "upstream_id": str(tributary["site_id"]), + "downstream_id": origin_site_id, + "label": f'Tributary: {tributary["site_id"]} -> {origin_site_id} | {distance_label} | {name}', + "kind": "tributary context", + "distance_km": tributary.get("distance_km"), + "name": name, + }) + + seen = set() + unique_pairs = [] + for pair in pairs: + if pair["key"] in seen: + continue + seen.add(pair["key"]) + unique_pairs.append(pair) + return unique_pairs[:max_pairs] + + +def _candidate_index_for_site(candidates, preferred_site_id, fallback_positions): + """Return the selector index for a preferred site or role fallback.""" + if preferred_site_id: + preferred_site_id = str(preferred_site_id) + for idx, candidate in enumerate(candidates): + if str(candidate.get("site_id")) == preferred_site_id: + return idx + for idx, candidate in enumerate(candidates): + if candidate.get("position") in fallback_positions: + return idx + return 0 + + +def _candidate_label_for_site(candidates, site_id): + """Return the dropdown label for a candidate site ID.""" + if site_id is None: + return None + site_id = str(site_id) + for candidate in candidates: + if str(candidate.get("site_id")) == site_id: + return candidate.get("label") + return None + + +def _default_candidate_label(candidates, preferred_site_id, fallback_positions): + """Return the dropdown label for a preferred site or role fallback.""" + preferred_label = _candidate_label_for_site(candidates, preferred_site_id) + if preferred_label: + return preferred_label + return candidates[_candidate_index_for_site(candidates, None, fallback_positions)]["label"] + + +def _selected_candidate_site_id(candidate_rows, selection_state): + """Return the site ID represented by a selected candidate table row.""" + try: + selected_rows = selection_state.get("selection", {}).get("rows", []) + except AttributeError: + selected_rows = getattr(getattr(selection_state, "selection", None), "rows", []) + if not selected_rows: + return None + row_index = selected_rows[0] + if row_index < 0 or row_index >= len(candidate_rows): + return None + return str(candidate_rows[row_index].get("Station")) + + +def _ensure_widget_value_is_valid(key, options): + """Clear stale widget values when candidate options change.""" + if key in st.session_state and st.session_state[key] not in options: + del st.session_state[key] + + +def _selectbox_kwargs_for_state(key, default_index, session_state): + """Avoid passing both explicit index and pre-set Streamlit widget state.""" + kwargs = {"key": key} + if key not in session_state: + kwargs["index"] = default_index + return kwargs + + +def _get_discharge_series(df): + """Return the primary discharge series from a fetched dataframe.""" + if "Discharge_cfs" in df.columns: + return df["Discharge_cfs"] + if "value" in df.columns: + return df["value"] + return df.select_dtypes(include="number").iloc[:, 0] + + +def _render_reach_map(up_info, dn_info, upstream_id, downstream_id, reach_km, search_km): + """Render selected reach flowlines and gage markers in the current Streamlit container.""" + if not (up_info and dn_info and up_info.get('latitude') and dn_info.get('latitude')): + st.info("Selected gage coordinates are unavailable for mapping.") + return + + import folium + from folium import Element + from streamlit_folium import st_folium + from hydrology.data.hyriver import get_flowlines + + up_lat, up_lon = float(up_info['latitude']), float(up_info['longitude']) + dn_lat, dn_lon = float(dn_info['latitude']), float(dn_info['longitude']) + center_lat = (up_lat + dn_lat) / 2 + center_lon = (up_lon + dn_lon) / 2 + + m = folium.Map( + location=[center_lat, center_lon], + zoom_start=11, + tiles=None, + max_bounds=True, + control_scale=True, + ) + folium.TileLayer( + "CartoDB dark_matter", + name="Base map", + no_wrap=True, + detect_retina=True, + ).add_to(m) + + flowline_distance = _flowline_distance_km(reach_km, search_km) + try: + flowlines = get_flowlines(downstream_id, distance_km=flowline_distance) + if flowlines is not None and not flowlines.empty: + folium.GeoJson( + flowlines.to_json(), + name="River network context", + style_function=lambda feature: _flowline_style(selected=False), + tooltip="NHDPlus river/tributary flowline", + ).add_to(m) + clipped_flowlines = _clip_flowlines_between_gages( + flowlines, + upstream_lat=up_lat, + upstream_lon=up_lon, + downstream_lat=dn_lat, + downstream_lon=dn_lon, + ) + if clipped_flowlines is not None and not clipped_flowlines.empty: + folium.GeoJson( + clipped_flowlines.to_json(), + name="Selected reach network", + style_function=lambda feature: _flowline_style(selected=True), + tooltip=f"Selected reach network: {upstream_id} -> {downstream_id}", + ).add_to(m) + else: + st.caption("Could not trace a connected NHD path between the selected gage points; showing river-network context only.") + else: + st.caption("River-network geometry was not available for this reach; showing gage locations only.") + except Exception as e: + logger.warning(f"Could not add reach flowlines: {e}") + st.caption("River-network geometry was not available for this reach; showing gage locations only.") + + folium.CircleMarker( + [up_lat, up_lon], radius=10, color='#2196F3', fill=True, + fillColor='#2196F3', fillOpacity=0.8, + tooltip=f"Upstream: {up_info.get('description', upstream_id)}" + ).add_to(m) + + folium.CircleMarker( + [dn_lat, dn_lon], radius=10, color='#FF9800', fill=True, + fillColor='#FF9800', fillOpacity=0.8, + tooltip=f"Downstream: {dn_info.get('description', downstream_id)}" + ).add_to(m) + + reach_bounds = _map_bounds_for_reach(None, None, up_lat, up_lon, dn_lat, dn_lon) + m.fit_bounds(reach_bounds, padding=(24, 24), max_zoom=13) + m.get_root().html.add_child(Element(_leaflet_fit_bounds_script(reach_bounds))) + st_folium( + m, + width=None, + height=520, + returned_objects=[], + key=_reach_map_component_key(upstream_id, downstream_id, reach_bounds), + ) + st.caption("Gold = selected reach network; blue = nearby river/tributary context. Blue marker = upstream gage; orange marker = downstream gage.") def show(): @@ -38,147 +731,250 @@ def show(): return st.header("Reach Analysis") - st.caption("Compare upstream and downstream stations to analyze gains, losses, and aquifer contributions") + st.caption("Select an upstream/downstream reach, then run gain/loss and baseflow analysis.") - # Each station gets its own independent search/filter - all_options = [f"{row['site_id']} - {str(row.get('description', ''))[:60]}" - for _, row in inventory_df.iterrows()] states = ["All States"] + sorted(FIPS_TO_STATE.values()) - col1, col2 = st.columns(2) - with col1: - st.subheader("Upstream Station") - up_search = st.text_input("Search", placeholder="River name or site ID...", - key="reach_up_search", label_visibility="collapsed") - up_state = st.selectbox("State", states, key="reach_up_state", label_visibility="collapsed") - up_filtered = _filter_inventory(inventory_df, up_search, up_state) - up_options = [f"{row['site_id']} - {str(row.get('description', ''))[:60]}" - for _, row in up_filtered.iterrows()] - if up_search or up_state != "All States": - st.caption(f"{len(up_options)} sites") - default_up_idx = 0 - for i, opt in enumerate(up_options): - if opt.startswith(DEFAULT_UPSTREAM): - default_up_idx = i - if up_options: - upstream_sel = st.selectbox("Upstream", up_options, index=default_up_idx, key="reach_upstream") - upstream_id = extract_site_id(upstream_sel) - up_info = get_cached_site_info(upstream_id) - if up_info: - st.caption(f"{up_info.get('description', '')}") - else: - st.warning("No sites match") - return + search_col1, search_col2, search_col3, search_col4, search_col5 = st.columns([1.05, 1.9, 3.0, 0.85, 1.15]) + with search_col1: + anchor_state = st.selectbox("State", states, key="reach_anchor_state") + with search_col2: + anchor_search = st.text_input( + "Find gage", + placeholder="River, station, or USGS ID...", + key="reach_anchor_search", + ) - with col2: - st.subheader("Downstream Station") - dn_search = st.text_input("Search", placeholder="River name or site ID...", - key="reach_dn_search", label_visibility="collapsed") - dn_state = st.selectbox("State", states, key="reach_dn_state", label_visibility="collapsed") - dn_filtered = _filter_inventory(inventory_df, dn_search, dn_state) - dn_options = [f"{row['site_id']} - {str(row.get('description', ''))[:60]}" - for _, row in dn_filtered.iterrows()] - if dn_search or dn_state != "All States": - st.caption(f"{len(dn_options)} sites") - default_dn_idx = 0 - for i, opt in enumerate(dn_options): - if opt.startswith(DEFAULT_DOWNSTREAM): - default_dn_idx = i - if dn_options: - downstream_sel = st.selectbox("Downstream", dn_options, index=default_dn_idx, key="reach_downstream") - downstream_id = extract_site_id(downstream_sel) - dn_info = get_cached_site_info(downstream_id) - if dn_info: - st.caption(f"{dn_info.get('description', '')}") - else: - st.warning("No sites match") - return + anchor_filtered = _filter_inventory(inventory_df, anchor_search, anchor_state) + anchor_options = [ + f"{row['site_id']} - {str(row.get('description', ''))[:80]}" + for _, row in anchor_filtered.iterrows() + ] + if not anchor_options: + st.warning("No gages match the current search") + return - st.markdown("---") + with search_col3: + anchor_sel = st.selectbox("Anchor gage", anchor_options, key="reach_anchor") + anchor_id = extract_site_id(anchor_sel) + anchor_info = get_cached_site_info(anchor_id) + with search_col4: + search_km = st.number_input( + "Km", + min_value=10, + max_value=300, + value=75, + step=5, + key="reach_nldi_search_km", + ) + with search_col5: + include_tributaries = st.toggle( + "Tributaries", + value=True, + key="reach_include_tributaries", + ) + find_gages = st.button("Find Reaches", width="stretch", key="reach_find_related") + if anchor_search or anchor_state != "All States": + st.caption(f"{len(anchor_options)} matching gages") + if anchor_info: + st.caption(anchor_info.get("description", "")) - # Date range - st.subheader("Date Range") - start_date, end_date = date_range_selector("reach", default_start=date(2000, 1, 1)) + related_key = f"reach_related_sites_{anchor_id}_{search_km}_{include_tributaries}" + if find_gages: + with st.spinner("Finding upstream and downstream gages on the river network..."): + st.session_state[related_key] = discover_related_sites( + anchor_id, + direction="both", + distance_km=float(search_km), + include_tributaries=include_tributaries, + max_sites=25, + ) - st.markdown("---") + discovered_related_sites = st.session_state.get(related_key, []) + related_sites, omitted_related_site_ids = _filter_related_sites_to_inventory( + anchor_id, + discovered_related_sites, + inventory_df, + ) + anchor_name = anchor_info.get("description", "Anchor gage") if anchor_info else "Anchor gage" + candidate_records = _build_reach_candidate_options(anchor_id, anchor_name, related_sites) + candidate_rows = _format_related_site_rows(anchor_id, related_sites) + reach_pairs = _build_recommended_reach_pairs(anchor_id, candidate_records) + selected_pair_key = _resolve_selected_pair_key(reach_pairs, st.session_state) + if selected_pair_key: + st.session_state["reach_selected_pair_key"] = selected_pair_key + selected_pair = next((pair for pair in reach_pairs if pair["key"] == selected_pair_key), None) - # Plot selection - default to showing key reach plots - st.subheader("Reach Analysis Plots") + if selected_pair: + upstream_id = selected_pair["upstream_id"] + downstream_id = selected_pair["downstream_id"] + else: + upstream_id = anchor_id + downstream_id = anchor_id + up_info = get_cached_site_info(upstream_id) + dn_info = get_cached_site_info(downstream_id) + estimated_reach_km = _estimate_reach_km(upstream_id, downstream_id, related_sites, anchor_id) + manual_reach_km = float(st.session_state.get("reach_length_km", 0.0) or 0.0) + reach_km = _resolve_reach_km(estimated_reach_km, manual_reach_km) reach_plot_options = {get_display_name(p): p for p in REACH_PLOTS if p in AVAILABLE_PLOTS} default_plots = ['reach_comparison', 'reach_index', 'seasonal_gain_loss'] default_display = [get_display_name(p) for p in default_plots if p in reach_plot_options.values()] - selected_display = st.multiselect( - "Select plots", - list(reach_plot_options.keys()), - default=default_display, - key="reach_plot_select" - ) - selected_plots = [reach_plot_options[d] for d in selected_display] + selected_display = default_display + layout_choice = "Auto" + dpi = 150 - # Show descriptions - with st.expander("Plot descriptions"): - for display_name, plot_key in reach_plot_options.items(): - info = AVAILABLE_PLOTS.get(plot_key, {}) - desc = info.get('description', '') if isinstance(info, dict) else '' - st.markdown(f"**{display_name}**: {desc}") - - st.markdown("---") - - # Reach map - show upstream/downstream stations - if up_info and dn_info and up_info.get('latitude') and dn_info.get('latitude'): - with st.expander("Reach Map", expanded=False): - import folium - from streamlit_folium import st_folium - - up_lat, up_lon = float(up_info['latitude']), float(up_info['longitude']) - dn_lat, dn_lon = float(dn_info['latitude']), float(dn_info['longitude']) - center_lat = (up_lat + dn_lat) / 2 - center_lon = (up_lon + dn_lon) / 2 - - m = folium.Map(location=[center_lat, center_lon], zoom_start=10, - tiles='CartoDB dark_matter') - - # Upstream marker (blue) - folium.CircleMarker( - [up_lat, up_lon], radius=10, color='#2196F3', fill=True, - fillColor='#2196F3', fillOpacity=0.8, - tooltip=f"Upstream: {up_info.get('description', upstream_id)}" - ).add_to(m) + candidate_col, map_col, summary_col = st.columns([1.05, 2.15, 1.0]) + with candidate_col: + st.subheader("Candidate Reaches") + if omitted_related_site_ids: + st.caption(f"{len(omitted_related_site_ids)} NLDI gages hidden outside HydroPlot inventory.") + if reach_pairs: + pair_labels = {pair["label"]: pair["key"] for pair in reach_pairs} + pair_label_options = list(pair_labels.keys()) + _ensure_widget_value_is_valid("reach_pair_select", pair_label_options) + selected_key = st.session_state.get("reach_selected_pair_key") - # Downstream marker (orange) - folium.CircleMarker( - [dn_lat, dn_lon], radius=10, color='#FF9800', fill=True, - fillColor='#FF9800', fillOpacity=0.8, - tooltip=f"Downstream: {dn_info.get('description', downstream_id)}" - ).add_to(m) + prev_col, count_col, next_col = st.columns([1, 1.1, 1]) + with prev_col: + if st.button("Previous", width="stretch", disabled=len(reach_pairs) <= 1, key="reach_pair_previous"): + selected_key = _cycle_pair_key(reach_pairs, selected_key, -1) + st.session_state["reach_selected_pair_key"] = selected_key + st.session_state["reach_pair_select"] = _pair_label_for_key(reach_pairs, selected_key) + with count_col: + current_index = [pair["key"] for pair in reach_pairs].index(st.session_state["reach_selected_pair_key"]) + 1 + st.caption(f"Reach {current_index} of {len(reach_pairs)}") + with next_col: + if st.button("Next", width="stretch", disabled=len(reach_pairs) <= 1, key="reach_pair_next"): + selected_key = _cycle_pair_key(reach_pairs, selected_key, 1) + st.session_state["reach_selected_pair_key"] = selected_key + st.session_state["reach_pair_select"] = _pair_label_for_key(reach_pairs, selected_key) - # River line connecting them - folium.PolyLine( - [[up_lat, up_lon], [dn_lat, dn_lon]], - color='#4FC3F7', weight=3, opacity=0.7, dash_array='10' - ).add_to(m) + selected_pair_label = ( + st.session_state.get("reach_pair_select") + or _pair_label_for_key(reach_pairs, st.session_state.get("reach_selected_pair_key")) + ) + chosen_label = st.selectbox( + "Selected candidate reach", + pair_label_options, + index=pair_label_options.index(selected_pair_label), + key="reach_pair_select", + ) + st.session_state["reach_selected_pair_key"] = pair_labels[chosen_label] + selected_pair = next(pair for pair in reach_pairs if pair["key"] == pair_labels[chosen_label]) + upstream_id = selected_pair["upstream_id"] + downstream_id = selected_pair["downstream_id"] + up_info = get_cached_site_info(upstream_id) + dn_info = get_cached_site_info(downstream_id) + estimated_reach_km = _estimate_reach_km(upstream_id, downstream_id, related_sites, anchor_id) + manual_reach_km = float(st.session_state.get("reach_length_km", 0.0) or 0.0) + reach_km = _resolve_reach_km(estimated_reach_km, manual_reach_km) + st.caption(f"{selected_pair['kind']} | {selected_pair.get('distance_km') or 'distance unknown'} km from anchor") + st.dataframe( + pd.DataFrame([ + { + "Reach": pair["label"], + "Kind": pair["kind"], + } + for pair in reach_pairs + ]), + width="stretch", + hide_index=True, + height=min(360, 38 + 34 * len(reach_pairs)), + ) + elif discovered_related_sites: + st.warning("NLDI found related gages, but none are available in the HydroPlot inventory.") + else: + st.info("Click Find Reaches to discover processable upstream/downstream candidates.") - st_folium(m, width=None, height=300, returned_objects=[]) - st.caption("Blue = upstream, Orange = downstream") + with map_col: + st.subheader("Reach Map") + _render_reach_map(up_info, dn_info, upstream_id, downstream_id, reach_km, search_km) - st.markdown("---") + with summary_col: + # Use styled workspace panel so entrance/hover polish from the premium CSS applies + if upstream_id == downstream_id: + render_workspace_panel( + "Selected Reach", + "Choose two different gages for a reach.", + [{"label": "Invalid pair", "state": "blocked"}], + ) + else: + chips = [] + if estimated_reach_km: + chips.append({"label": f"{estimated_reach_km:.1f} km network", "state": "ready"}) + elif manual_reach_km: + chips.append({"label": f"{manual_reach_km:.1f} km manual", "state": "limited"}) + chips.append({"label": f"{len(candidate_records)} candidates", "state": "ready"}) + render_workspace_panel( + "Selected Reach", + f"{upstream_id} \u2192 {downstream_id}", + chips, + ) + generate = st.button( + "Run Analysis", + type="primary", + width="stretch", + key="gen_reach", + disabled=upstream_id == downstream_id, + ) - # Layout and generate - col_layout, col_dpi, col_btn = st.columns([2, 1, 2]) - with col_layout: - layout_choice = st.selectbox("Layout", ["Auto", "Vertical", "Grid 2x3"], key="reach_layout") - with col_dpi: - dpi = st.number_input("DPI", min_value=72, max_value=300, value=150, key="reach_dpi") - with col_btn: - st.markdown("<br>", unsafe_allow_html=True) - generate = st.button("Generate Reach Analysis", type="primary", width="stretch", key="gen_reach") + with st.expander("Analysis settings", expanded=False): + settings_col1, settings_col2, settings_col3 = st.columns([1.2, 1.7, 0.7]) + with settings_col1: + start_date, end_date = date_range_selector("reach", default_start=date(2000, 1, 1)) + with settings_col2: + selected_display = st.multiselect( + "Plots", + list(reach_plot_options.keys()), + default=default_display, + key="reach_plot_select" + ) + with settings_col3: + layout_choice = st.selectbox("Layout", ["Auto", "Vertical", "Grid 2x3"], key="reach_layout") + dpi = st.number_input("DPI", min_value=72, max_value=300, value=150, key="reach_dpi") + with st.expander("Plot descriptions", expanded=False): + for display_name, plot_key in reach_plot_options.items(): + info = AVAILABLE_PLOTS.get(plot_key, {}) + desc = info.get('description', '') if isinstance(info, dict) else '' + st.markdown(f"**{display_name}**: {desc}") + selected_plots = [reach_plot_options[d] for d in selected_display] + + with st.expander("Candidate gage details", expanded=False): + if omitted_related_site_ids: + st.caption( + f"{len(omitted_related_site_ids)} NLDI gages were hidden because they are not in the HydroPlot inventory for this app." + ) + if related_sites: + st.dataframe(pd.DataFrame(candidate_rows), width="stretch", hide_index=True) + elif discovered_related_sites: + st.warning("NLDI found related gages, but none are available in the HydroPlot inventory for this dashboard.") + else: + st.info("Use Find Reaches to discover likely upstream/downstream candidates for the selected anchor gage.") + + with st.expander("Advanced reach length override", expanded=False): + manual_reach_km = st.number_input( + "Manual reach length km", + min_value=0.0, + max_value=1000.0, + value=manual_reach_km, + step=0.1, + help="Optional. Used only when the network length cannot be inferred.", + key="reach_length_km", + ) + if estimated_reach_km: + st.caption("Network-inferred length is used for cfs/km. Manual value is ignored while an inferred length is available.") + else: + st.caption("Optional fallback for cfs/km when related gage distances do not define the selected reach.") if generate: if not selected_plots: st.warning("Select at least one plot") return + if upstream_id == downstream_id: + st.warning("Choose two different gages before generating reach analysis.") + return start_str = start_date.strftime('%Y-%m-%d') end_str = end_date.strftime('%Y-%m-%d') @@ -219,6 +1015,23 @@ def show(): with col2: st.metric("Downstream", f"{len(df_downstream):,} days", delta=dn_desc, delta_color="off") + upstream_q = _get_discharge_series(df_upstream) + downstream_q = _get_discharge_series(df_downstream) + reach_row = _build_reach_summary_row(upstream_id, downstream_id, upstream_q, downstream_q, reach_km=reach_km) + length_source = "network" if estimated_reach_km else "manual" if reach_km else "missing" + reach_interpretation = _build_reach_interpretation(reach_row, reach_km=reach_km, length_source=length_source) + # Styled wrapper so new card entrance / hover / depth polish applies uniformly + render_workspace_panel( + "Automated Reach Summary", + f"{reach_interpretation.get('Finding', 'Reach result')} — {reach_interpretation.get('Interpretation', '')[:120]}", + [{"label": reach_interpretation.get("Confidence", "n/a"), "state": "ready" if reach_interpretation.get("Confidence") == "high" else "limited"}], + ) + # Keep the compact table for the detailed numbers (existing behavior) + st.dataframe(pd.DataFrame([reach_interpretation]), width="stretch", hide_index=True) + with st.expander("Reach details", expanded=False): + st.dataframe(pd.DataFrame(_format_reach_chain([upstream_id, downstream_id])), width="stretch", hide_index=True) + st.dataframe(pd.DataFrame([reach_row]), width="stretch", hide_index=True) + st.markdown("---") # Generate plots @@ -280,7 +1093,11 @@ def show(): # Baseflow separation waterfall (interactive Plotly) st.markdown("---") - st.subheader("Baseflow Separation") + render_workspace_panel( + "Baseflow Separation", + "Interactive view of quickflow vs baseflow contribution across the paired record.", + [{"label": "Plotly", "state": "ready"}], + ) show_waterfall = st.checkbox( "Show Baseflow Waterfall", value=True, key="show_bf_waterfall" ) diff --git a/hydrology/app/page_modules/single_analysis.py b/hydrology/app/page_modules/single_analysis.py index 9c2f855..c1451ed 100644 --- a/hydrology/app/page_modules/single_analysis.py +++ b/hydrology/app/page_modules/single_analysis.py @@ -18,13 +18,23 @@ render_site_header, render_availability_badges, render_metric_cards, render_plot_capability_board, render_insight_board ) -from hydrology.app.interpretation import summarize_flow_context, summarize_recommendations +from hydrology.app.interpretation import ( + summarize_flow_context, + summarize_recommendations, + build_plot_analysis_report, + build_interactive_chart_brief, +) from hydrology.app.plot_config import SINGLE_SITE_PLOTS, resolve_generated_plots +from hydrology.app.page_modules.indicators import render_site_indicators from hydrology.visualization import create_multi_plot, PlotLayout from hydrology.visualization.interactive import ( - interactive_hydrograph, interactive_fdc, + interactive_hydrograph, interactive_fdc, interactive_rating_curve, raster_hydrograph, percentile_bands_hydrograph) from hydrology.core import DEFAULT_DISCHARGE_CODE +from hydrology.analysis.stage_discharge import ( + fit_best_rating_curve, + evaluate_rating_params, +) @st.cache_data(ttl=3600, show_spinner=False) @@ -33,6 +43,206 @@ def _load_site_data(site_id, lat, lon, start_str, end_str): return process_site_data(site_id, lat, lon, start_str, end_str) +def _render_rating_curve_workshop( + df_q: pd.DataFrame, + *, + site_id: str, + site_desc: str, +) -> None: + """Sleek interactive stage–discharge fit: adjust A, B, H0 and see the curve update.""" + import math + + if df_q is None or df_q.empty: + return + if "Gage_Height_ft" not in df_q.columns or "Discharge_cfs" not in df_q.columns: + return + + pairs = df_q[["Gage_Height_ft", "Discharge_cfs"]].dropna() + pairs = pairs[(pairs["Gage_Height_ft"] > 0) & (pairs["Discharge_cfs"] > 0)] + if len(pairs) < 10: + return + + st.markdown("---") + st.subheader("Rating curve workshop") + st.caption( + "Tune the stage–discharge model live. **Red** = your parameters · " + "**Teal dashed** = auto-fit reference. Residuals show where the rating is biased." + ) + + stage = pairs["Gage_Height_ft"] + discharge = pairs["Discharge_cfs"] + auto = fit_best_rating_curve(stage, discharge, min_points=10) + if auto.get("model") == "none" or not pd.notna(auto.get("A")): + st.info("Could not auto-fit a rating curve for this period.") + return + + state_key = f"rating_ws_{site_id}" + auto_A = float(auto["A"]) + auto_B = float(auto["B"]) + auto_H0 = float(auto.get("H0") or 0.0) + auto_r2 = float(auto["R2"]) if pd.notna(auto.get("R2")) else float("nan") + auto_eq = auto.get("equation", "") + + if state_key not in st.session_state: + st.session_state[state_key] = { + "A": auto_A, + "B": auto_B, + "H0": auto_H0, + "use_offset": auto.get("model") == "offset_powerlaw", + "ver": 0, + } + ws = st.session_state[state_key] + # Keep auto baseline current for this period + ws["auto_A"] = auto_A + ws["auto_B"] = auto_B + ws["auto_H0"] = auto_H0 + ws["auto_r2"] = auto_r2 + ws["auto_eq"] = auto_eq + ver = int(ws.get("ver", 0)) + + c_left, c_right = st.columns([1, 1.55], gap="large") + + with c_left: + st.markdown("##### Parameters") + use_offset = st.toggle( + "Offset model Q = A·(H − H₀)ᴮ", + value=bool(ws.get("use_offset", True)), + key=f"{state_key}_offset_{ver}", + help="H₀ is the effective zero-flow stage when gage zero ≠ control elevation.", + ) + ws["use_offset"] = use_offset + + A_ref = max(float(ws.get("A", auto_A)), 1e-9) + log_A_ref = math.log10(max(auto_A, 1e-9)) + log_A_min = log_A_ref - 2.0 + log_A_max = log_A_ref + 2.0 + cur_log_A = min(max(math.log10(A_ref), log_A_min), log_A_max) + + log_A = st.slider( + "A (scale, log₁₀)", + min_value=float(log_A_min), + max_value=float(log_A_max), + value=float(cur_log_A), + step=0.01, + key=f"{state_key}_logA_{ver}", + help="Coefficient in front of the power law (log scale for wide range).", + ) + A = 10.0 ** log_A + st.caption(f"A = **{A:.4g}**") + + B = st.slider( + "B (exponent)", + min_value=0.15, + max_value=4.0, + value=float(min(max(float(ws.get("B", auto_B)), 0.15), 4.0)), + step=0.01, + key=f"{state_key}_B_{ver}", + help="Control shape. ~1.5–2.5 is common for open channels.", + ) + + H0 = 0.0 + if use_offset: + H_min = float(stage.min()) + H_max = float(stage.max()) + h0_hi = H_min - 0.01 + span = max(H_max - H_min, 0.5) + h0_lo = min(H_min - span - 2.0, h0_hi - 0.5) + h0_default = float(min(max(float(ws.get("H0", auto_H0)), h0_lo), h0_hi)) + H0 = st.slider( + "H₀ (zero-flow stage, ft)", + min_value=float(h0_lo), + max_value=float(h0_hi), + value=h0_default, + step=0.01, + key=f"{state_key}_H0_{ver}", + help="Stage of zero discharge. Raising H₀ steepens the low end.", + ) + ws["A"] = A + ws["B"] = B + ws["H0"] = H0 + + b1, b2 = st.columns(2) + with b1: + if st.button("↩ Auto-fit", key=f"{state_key}_reset", use_container_width=True): + ws["A"] = auto_A + ws["B"] = auto_B + ws["H0"] = auto_H0 + ws["use_offset"] = abs(auto_H0) > 1e-6 or auto.get("model") == "offset_powerlaw" + ws["ver"] = ver + 1 + st.rerun() + with b2: + st.caption(f"Auto: `{auto_eq}`") + + scored = evaluate_rating_params(stage, discharge, A, B, H0) + m1, m2, m3, m4 = st.columns(4) + r2 = scored.get("R2") + rmse = scored.get("rmse") + nse = scored.get("nash_sutcliffe") + bias = scored.get("bias") + m1.metric( + "R²", + f"{r2:.3f}" if pd.notna(r2) else "—", + help="Fraction of Q variance explained by the rating. ~1.0 = tight fit; <0.7 often means hysteresis, ice, or a shifted control.", + ) + m2.metric( + "RMSE", + f"{rmse:,.0f}" if pd.notna(rmse) else "—", + help="Root-mean-square residual in cfs. Lower is better; sensitive to a few big misses at high stage.", + ) + m3.metric( + "NSE", + f"{nse:.3f}" if pd.notna(nse) else "—", + help="Nash–Sutcliffe efficiency. 1 = perfect; 0 = no better than using mean Q; negative = worse than the mean.", + ) + m4.metric( + "Bias", + f"{bias:+,.0f}" if pd.notna(bias) else "—", + help="Mean (Q_fit − Q_obs) in cfs. Positive = rating overpredicts; negative = underpredicts.", + ) + + if pd.notna(r2) and pd.notna(auto_r2): + delta = r2 - auto_r2 + if delta >= 0.005: + st.success(f"Your fit beats auto-fit by **ΔR² = {delta:+.3f}**") + elif delta <= -0.02: + st.warning(f"Auto-fit is stronger (ΔR² = {delta:+.3f}). Try ↩ Auto-fit.") + else: + st.info(f"Close to auto-fit (ΔR² = {delta:+.3f})") + + st.markdown(f"**Equation:** `{scored.get('equation', '—')}`") + st.caption( + f"{scored.get('n_points', 0):,} pairs · " + "Daily mean Q (00060) vs stage (00065) — empirical consistency check, " + "not a survey-grade rating table." + ) + + with c_right: + color_by = st.radio( + "Color points", + options=["season", "year"], + format_func=lambda x: "Season (DJF/MAM/JJA/SON)" if x == "season" else "Year", + horizontal=True, + key=f"{state_key}_color", + ) + show_resid = st.checkbox("Show residuals", value=True, key=f"{state_key}_resid") + default_log = bool((discharge.max() / max(float(discharge.min()), 1e-9)) >= 40) + log_q = st.checkbox("Log discharge axis", value=default_log, key=f"{state_key}_logq") + + fig = interactive_rating_curve( + df_q, + A=A, + B=B, + H0=H0, + title=f"{site_desc} — rating workshop", + show_auto_fit=True, + auto_fit=auto, + show_residuals=show_resid, + log_q=bool(log_q), + color_by=color_by, + ) + st.plotly_chart(fig, width="stretch", key=f"plotly_rating_{site_id}") + + def _render_analysis_readiness(data, has_stage: bool, climate_info): """Show visible plot readiness before users open deeper controls.""" df_q = data.get('df_q') if data else None @@ -52,29 +262,45 @@ def _render_analysis_readiness(data, has_stage: bool, climate_info): "body": f"{discharge_days:,} daily observations for hydrographs, duration curves, anomalies, and seasonal context.", "status": "Ready" if discharge_days else "Unavailable", "state": "ready" if discharge_days else "blocked", + "help": ( + "Unlocks interactive hydrograph, flow-duration curve, Q10/Q50/Q90 stats, " + "and most static plots. If blocked, widen the date range or pick another gage." + ), }, { "title": "Climate Links", "body": "Temperature and precipitation overlays, lag response, SPI, and correlation views.", "status": climate_status, "state": "ready" if has_climate else "limited", + "help": ( + "Merged precip/temp near this gage. Ready = co-plotted climate and SPI context " + "are usable. Distant station = signals may lag or smear local storms." + ), }, { "title": "Stage + Rating", "body": "Dual-axis stage overlay and stage-discharge rating curve when gage height exists.", "status": "Ready" if has_stage else "Needs gage height", "state": "ready" if has_stage else "blocked", + "help": ( + "Needs daily gage height (00065). When ready, use stage overlay on the " + "hydrograph and the Rating curve workshop (A, B, H₀) further down the page." + ), }, { "title": "Frequency", "body": "Flood frequency runs on demand from annual peak records so it does not slow page load.", "status": "On demand", "state": "limited", + "help": ( + "Not auto-run. Open Frequency Analysis (expander) when you need return-period " + "estimates. Best with longer records; short periods are exploratory only." + ), }, ] st.subheader("Analysis Readiness") - st.caption("Plot availability is based on the selected site, date range, and linked weather station.") + st.caption("What this site/period can support. Hover or tap a tile for details.") render_plot_capability_board(cards) @@ -90,7 +316,7 @@ def _render_hydrologic_summary(data, has_stage: bool, climate_info): record_years = (df_q.index.max() - df_q.index.min()).days / 365.25 st.subheader("Hydrologic Summary") - st.caption("Automated context from the selected site's historical daily record.") + st.caption("Automated context for this period. Hover (desktop) or tap (mobile) a tile for detail.") render_insight_board(summarize_flow_context(df_q)) with st.expander("Recommended next views", expanded=False): @@ -104,6 +330,20 @@ def _render_hydrologic_summary(data, has_stage: bool, climate_info): st.caption(f"Climate-linked recommendations use {station}.") +def _format_frequency_diagnostics(diagnostics: pd.DataFrame) -> pd.DataFrame: + """Format flood-frequency diagnostics for dashboard display.""" + display = diagnostics[["observed_flow_cfs", "fitted_flow_cfs", "return_period"]].copy() + display = display.rename(columns={ + "observed_flow_cfs": "Observed flow", + "fitted_flow_cfs": "Fitted flow", + "return_period": "Return period", + }) + for col in ["Observed flow", "Fitted flow"]: + display[col] = display[col].apply(lambda value: f"{value:,.0f}" if pd.notna(value) else "N/A") + display["Return period"] = display["Return period"].apply(lambda value: f"{value:.1f} yr") + return display + + def show(): """Render the Single Analysis page.""" inventory_df = get_inventory() @@ -152,7 +392,8 @@ def show(): has_stage = 'Gage_Height_ft' in data['df_q'].columns if data['df_q'] is not None else False climate_info = get_weather_station_info(float(lat), float(lon)) if lat and lon else None render_availability_badges(True, has_stage, climate_info) - render_metric_cards(data['df_q'], data['df_merged']) + # Record coverage only at top — duration quantiles live next to the FDC + render_metric_cards(data['df_q'], data['df_merged'], section="record") st.markdown("---") _render_hydrologic_summary(data, has_stage, climate_info) @@ -171,10 +412,10 @@ def show(): help="Changes the hydrograph time scale without refetching the selected site data.") with col_nwm: show_nwm = st.checkbox( - "NWM forecast overlay", + "NWM recent analysis overlay", value=False, key="nwm_overlay", - help="Adds NOAA National Water Model streamflow where a matching forecast record is available.", + help="Adds recent NOAA National Water Model analysis/assimilation streamflow when a matching reach is available.", ) with col_stage: show_stage = st.checkbox( @@ -212,7 +453,7 @@ def show(): line=dict(color='#ff7f0e', width=2, dash='dot'))) st.caption(f"NWM: NSE={nwm_result.nash_sutcliffe:.2f}, RMSE={nwm_result.rmse:.0f} cfs") else: - st.caption("NWM data not available for this site/period") + st.caption("NWM recent analysis was not available for this site/period") except Exception as e: st.caption(f"NWM overlay unavailable: {str(e)[:60]}") @@ -237,6 +478,10 @@ def show(): st.plotly_chart(fig_hydro, width="stretch", key="plotly_hydro") + # Duration stats sit with the FDC (not at page top) + st.markdown("---") + render_metric_cards(data["df_q"], data.get("df_merged"), section="duration") + # Flow Duration Curve koehler = st.checkbox( "Koehler (2025) dQ/dt coloring", @@ -250,9 +495,29 @@ def show(): color_by_dqdt=koehler) st.plotly_chart(fig_fdc, width="stretch", key="plotly_fdc") + # Plain-language read of interactive charts + with st.container(): + st.markdown("##### What these interactive charts are saying") + st.markdown( + build_interactive_chart_brief(data.get("df_q"), data.get("df_merged")) + ) + + # ── Interactive rating-curve workshop ── + if has_stage: + _render_rating_curve_workshop(data["df_q"], site_id=site_id, site_desc=desc) + # CSV download render_data_download(data['df_q'], filename_prefix=site_id) + with st.expander("Drought & Baseflow Indicators", expanded=False): + st.caption("SRI, precipitation SPI, baseflow proxy, and seasonal anomaly for the selected gage.") + indicator_loaded_key = f"site_indicators_loaded_{site_id}" + indicator_button_key = f"load_site_indicators_{site_id}" + if st.button("Load Indicators", type="primary", key=indicator_button_key): + st.session_state[indicator_loaded_key] = True + if st.session_state.get(indicator_loaded_key): + render_site_indicators(site_id, site_info) + # Advanced visualizations (opt-in) st.markdown("---") st.subheader("Advanced Visualizations") @@ -284,7 +549,8 @@ def show(): if st.button("Run Frequency Analysis", type="primary", key="gen_freq"): from hydrology.data.usgs import fetch_peak_streamflow from hydrology.analysis.frequency import ( - fit_flood_frequency, estimate_return_periods, get_plotting_positions) + fit_flood_frequency, estimate_return_periods, + flood_frequency_diagnostics, get_plotting_positions) from hydrology.visualization.interactive import interactive_return_period with st.spinner("Fetching peak streamflow data..."): @@ -325,6 +591,16 @@ def show(): ) st.dataframe(display_rp, width="stretch", hide_index=True) + diagnostics = flood_frequency_diagnostics(peak_values, distribution="lp3") + if not diagnostics.empty: + st.subheader("LP3 Diagnostic Table") + st.dataframe( + _format_frequency_diagnostics(diagnostics).head(25), + width="stretch", + hide_index=True, + ) + st.caption("Observed plotting positions compared with fitted LP3 quantiles.") + # Model comparison table st.subheader("Distribution Comparison") model_rows = [] @@ -399,3 +675,25 @@ def show(): st.pyplot(fig) render_export_buttons(fig, site_id, dpi) plt.close(fig) + + # Text analysis of the data behind the generated plots + st.markdown("---") + st.subheader("Automated plot analysis") + st.caption( + "Plain-language screening narrative from the selected period and plot set." + ) + report = build_plot_analysis_report( + site_id=site_id, + site_desc=desc, + plot_keys=static_plots, + df_q=data.get("df_q"), + df_merged=data.get("df_merged"), + ) + st.markdown(report) + st.download_button( + "Download analysis as Markdown", + data=report, + file_name=f"{site_id}_plot_analysis.md", + mime="text/markdown", + key=f"dl_plot_analysis_{site_id}", + ) diff --git a/hydrology/app/page_modules/watershed.py b/hydrology/app/page_modules/watershed.py new file mode 100644 index 0000000..636d725 --- /dev/null +++ b/hydrology/app/page_modules/watershed.py @@ -0,0 +1,19 @@ +"""Watershed and basin context page.""" + +import streamlit as st + +from hydrology.app.shared import get_inventory +from hydrology.app.page_modules.advanced import _watershed_view + + +def show(): + """Render watershed inventory, basin boundaries, and basin characteristics.""" + st.header("Watershed") + st.caption("Inspect basin boundaries, dams, land cover, and HUC inventory context.") + + inventory_df = get_inventory() + if inventory_df.empty: + st.error("Could not load site inventory") + return + + _watershed_view(inventory_df) diff --git a/hydrology/app/shared.py b/hydrology/app/shared.py index 9859b59..9649de9 100644 --- a/hydrology/app/shared.py +++ b/hydrology/app/shared.py @@ -25,7 +25,11 @@ fetch_instantaneous_values, DEFAULT_PARAM_DISCHARGE, DEFAULT_PARAM_STAGE, fetch_current_conditions, fetch_daily_percentiles, classify_condition ) -from hydrology.data.climate import fetch_climate_data, fetch_nearest_station_info +from hydrology.data.climate import ( + fetch_climate_data, + fetch_nearest_station_info, + fetch_open_meteo_climate, +) from hydrology.data.nwm import NWMClient, compare_nwm_usgs, get_forecast_skill from hydrology.analysis.alerts import ( AlertMonitor, AlertThreshold, create_flood_alert, create_low_flow_alert @@ -279,22 +283,6 @@ def find_availability_windows(site_id: str, param_cd: str, check_iv: bool = Fals except Exception: pass - # Fallback to the local filtered inventory for legacy long-record sites - # (e.g. 12510500 Kiona, WA — real POR since ~1905 but USGS series catalog - # or the snapshot often only reports 1948). This mitigates the "stale - # inventory ignored" problem created by prior agent changes without - # discarding the primary live catalog path. - try: - site_info = get_site_info(site_id) - if site_info: - begin = site_info.get('begin_date') - if begin: - start_year = int(str(begin)[:4]) - if start_year < current_year - 1: - return [(start_year, "present")] - except Exception: - pass - return None @@ -395,66 +383,161 @@ def fetch_discharge_data(site_id: str, param_cd: str, start_str: str, end_str: s @st.cache_data(ttl=3600, show_spinner=False) -def fetch_climate_cached(lat: float, lon: float, start_str: str, end_str: str, site_id: str | None = None): +def fetch_climate_cached_result( + lat: float, + lon: float, + start_str: str, + end_str: str, + site_id: str | None = None, + include_temp: bool = True, + include_precip: bool = True, +): + """Fetch normalized climate data with source metadata. + + Order of preference (reliability-first for SPI/dashboard): + 1. Open-Meteo Historical (no key, gridded, strong modern coverage) + 2. Meteostat multi-station fallback (skips dead nearest stations) + 3. Daymet (needs NASA Earthdata; optional) + + Set HYDRO_PREFER_DAYMET=1 to try Daymet earlier when credentials exist. """ - Fetch climate data with per-gage source awareness and multiple providers. + import os - Sourcing order (chosen for historical depth + reliability + low friction): - 1. Open-Meteo Historical (excellent coverage, no key, very stable) - 2. Meteostat (station-based) - 3. Daymet (gridded, optional heavy dependency) + start_dt = datetime.strptime(start_str, "%Y-%m-%d") + end_dt = datetime.strptime(end_str, "%Y-%m-%d") - Returns structured metadata so the rest of the app can behave dynamically - per gage when historical climate data is poor or missing. - """ - start_dt = datetime.strptime(start_str, '%Y-%m-%d') - end_dt = datetime.strptime(end_str, '%Y-%m-%d') + prefer_daymet = os.environ.get("HYDRO_PREFER_DAYMET", "").strip().lower() in ( + "1", + "true", + "yes", + "on", + ) - # === Provider 1: Open-Meteo Historical (preferred for historical depth) === - try: - from hydrology.data.climate import fetch_open_meteo_climate - om = fetch_open_meteo_climate(lat, lon, start_str, end_str) - om_norm = normalize_climate_columns(om) - if om_norm is not None and not om_norm.empty: + def _try_open_meteo(): + try: + om = fetch_open_meteo_climate(lat, lon, start_str, end_str) + normalized = normalize_climate_columns(om) + if normalized is None or normalized.empty: + return None + # Drop unused columns if only precip/temp requested + cols = [] + if include_temp and "Temp_C" in normalized.columns: + cols.append("Temp_C") + if include_precip and "Precip_mm" in normalized.columns: + cols.append("Precip_mm") + if not cols: + return None + data = normalized[cols] return { - 'data': om_norm, - 'source': 'open-meteo', - 'quality': 'good' if len(om_norm) > 30 else 'partial' + "data": data, + "source": "Open-Meteo", + "message": ( + "Loaded gridded Open-Meteo historical climate at the gage location " + "(no station gap / no API key)." + ), + "station_name": "Open-Meteo grid", + "station_distance_km": 0.0, } - except Exception as e: - logger.info(f"Open-Meteo unavailable for {site_id or (lat,lon)}: {e}") - - # === Provider 2: Meteostat === - station_climate = normalize_climate_columns(fetch_climate_data( - lat, lon, - pd.Timestamp(start_dt), - pd.Timestamp(end_dt), - include_temp=True, - include_precip=True - )) - if station_climate is not None and not station_climate.empty: - return { - 'data': station_climate, - 'source': 'meteostat', - 'quality': 'good' if len(station_climate) > 30 else 'partial' - } - - # === Provider 3: Daymet (optional) === - if site_id: + except Exception as e: + logger.info(f"Open-Meteo unavailable for {site_id or (lat, lon)}: {e}") + return None + + def _try_daymet(): + if not site_id: + return None + variables = [] + if include_precip: + variables.append("prcp") + if include_temp: + variables.extend(["tmin", "tmax"]) try: from hydrology.data.hyriver import get_daymet_climate - daymet = get_daymet_climate(site_id, start_str, end_str, variables=['prcp', 'tmin', 'tmax']) + + daymet = get_daymet_climate( + site_id, start_str, end_str, variables=variables or None + ) normalized = normalize_climate_columns(daymet) if normalized is not None and not normalized.empty: return { - 'data': normalized, - 'source': 'daymet', - 'quality': 'good' if len(normalized) > 30 else 'partial' + "data": normalized, + "source": "Daymet", + "message": "Loaded gridded Daymet climate data for the selected gage.", } except Exception as e: logger.info(f"Daymet climate unavailable for {site_id}: {e}") + return None + + def _try_meteostat(): + station_climate = normalize_climate_columns( + fetch_climate_data( + lat, + lon, + pd.Timestamp(start_dt), + pd.Timestamp(end_dt), + include_temp=include_temp, + include_precip=include_precip, + ) + ) + if station_climate is not None and not station_climate.empty: + station = None + try: + station = fetch_nearest_station_info(lat, lon, prefer_recent=True) + except Exception: + pass + dist = None + name = None + if station: + dist = station.get("distance_km") + name = station.get("name") + msg = "Loaded climate data from a Meteostat station with period coverage." + if name and dist is not None: + msg = ( + f"Loaded Meteostat station “{name}” " + f"({float(dist):.1f} km from the gage)." + ) + elif name: + msg = f"Loaded Meteostat station “{name}”." + return { + "data": station_climate, + "source": "Meteostat", + "message": msg, + "station_name": name, + "station_distance_km": dist, + } + return None + + if prefer_daymet: + daymet_result = _try_daymet() + if daymet_result: + return daymet_result + + # Primary: Open-Meteo (fixes sites where nearest Meteostat station is historical-only) + om = _try_open_meteo() + if om: + return om - return {'data': None, 'source': 'none', 'quality': 'none'} + meteo = _try_meteostat() + if meteo: + return meteo + + daymet_result = _try_daymet() + if daymet_result: + return daymet_result + + return { + "data": None, + "source": "Unavailable", + "message": ( + "Could not load climate data from Open-Meteo, Meteostat, or Daymet " + "for this site and date range." + ), + } + + +@st.cache_data(ttl=3600, show_spinner=False) +def fetch_climate_cached(lat: float, lon: float, start_str: str, end_str: str, site_id: str | None = None): + """Fetch normalized climate data - cached.""" + return fetch_climate_cached_result(lat, lon, start_str, end_str, site_id)["data"] def normalize_climate_columns(df: pd.DataFrame | None) -> pd.DataFrame | None: @@ -784,10 +867,7 @@ def process_site_data(site_id: str, lat: float, lon: float, start_str: str, end_ except Exception as e: logger.info(f"Stage data not available for {site_id}: {e}") - climate_result = fetch_climate_cached(lat, lon, start_str, end_str, site_id) - df_climate = climate_result.get('data') if isinstance(climate_result, dict) else climate_result - climate_source = climate_result.get('source', 'none') if isinstance(climate_result, dict) else ('meteostat' if df_climate is not None else 'none') - climate_quality = climate_result.get('quality', 'none') if isinstance(climate_result, dict) else ('good' if df_climate is not None else 'none') + df_climate = fetch_climate_cached(lat, lon, start_str, end_str, site_id) df_merged = None analysis_results = None @@ -799,7 +879,7 @@ def process_site_data(site_id: str, lat: float, lon: float, start_str: str, end_ df_merged = pd.merge(df_q, df_climate, left_index=True, right_index=True, how='inner') if df_merged.empty: - logger.warning(f"Merge produced empty result for {site_id}. df_q: {len(df_q)} rows, df_climate: {len(df_climate)} rows") + logger.warning(f"Merge produced empty result. df_q: {len(df_q)} rows, df_climate: {len(df_climate)} rows") df_merged = None else: analysis_results = analyze_correlation(df_merged) @@ -819,11 +899,7 @@ def process_site_data(site_id: str, lat: float, lon: float, start_str: str, end_ 'climate_count': len(df_climate) if df_climate is not None else 0, 'merged_count': len(df_merged) if df_merged is not None else 0, 'discharge_coverage': discharge_coverage, - 'stage_coverage': stage_coverage, - # New per-gage dynamic climate metadata (makes climate behavior gage-specific) - 'climate_source': climate_source, - 'climate_quality': climate_quality, - 'climate_available': df_climate is not None and not (df_climate.empty if df_climate is not None else True), + 'stage_coverage': stage_coverage } diff --git a/hydrology/app/streamlit_app.py b/hydrology/app/streamlit_app.py index bbba37d..65ff1aa 100644 --- a/hydrology/app/streamlit_app.py +++ b/hydrology/app/streamlit_app.py @@ -1,99 +1,15 @@ """ -Hydrology Analysis Dashboard - Multipage App Entry Point +Hydrology Analysis Dashboard entrypoint (compat with existing process lines). -Run with: streamlit run hydrology/app/app.py +Preferred: streamlit run hydrology/app/app.py +Also works: streamlit run hydrology/app/streamlit_app.py """ import sys from pathlib import Path -# Ensure the hydrology package is importable +# Ensure the hydrology package is importable when launched as a script path sys.path.insert(0, str(Path(__file__).parent.parent.parent)) -import streamlit as st - -st.set_page_config(page_title="Hydrology Analysis", page_icon="\U0001f4a7", layout="wide") - -from hydrology.app.styles import ( - apply_custom_css, render_footer, render_dashboard_hero, - render_workflow_strip, render_main_nav -) -from hydrology.app.shared import get_inventory -from hydrology.visualization.plots import AVAILABLE_PLOTS - -# Page imports -from hydrology.app.page_modules import overview, single_analysis, comparisons, reach_analysis, alerts, advanced, indicators - -apply_custom_css() -render_dashboard_hero( - "HydroPlot analysis dashboard", - "Find stations, analyze one site, compare gages, and run current hydrologic checks from one workspace.", -) -render_workflow_strip() - -# Background cache warmup — pre-fetch data so first user doesn't wait -if "warmup_started" not in st.session_state: - st.session_state["warmup_started"] = True - import threading - - def _warmup(): - try: - from hydrology.data.usgs import fetch_current_conditions, fetch_daily_values - from hydrology.data.usgs import DEFAULT_PARAM_DISCHARGE - - # Warm priority sites - from datetime import datetime, timedelta - priority = ["12422500", "12424000", "12419000"] - fetch_current_conditions(priority) - end = datetime.now().strftime('%Y-%m-%d') - start = (datetime.now() - timedelta(days=365)).strftime('%Y-%m-%d') - for sid in priority: - try: - fetch_daily_values(sid, param_cd=DEFAULT_PARAM_DISCHARGE, - start_date=start, end_date=end) - except Exception: - pass - except Exception: - pass # Warmup is best-effort - - threading.Thread(target=_warmup, daemon=True).start() - -# Define pages with grouping -_single_analysis_page = st.Page(single_analysis.show, title="Single Analysis", icon="📈", url_path="single-analysis") -_overview_page = st.Page(overview.show, title="Overview", icon="\U0001f4ca", default=True, url_path="overview") -_comparisons_page = st.Page(comparisons.show, title="Comparisons", icon="\U0001f504", url_path="comparisons") -_reach_page = st.Page(reach_analysis.show, title="Reach Analysis", icon="\U0001f30a", url_path="reach-analysis") -_alerts_page = st.Page(alerts.show, title="Alerts", icon="\U0001f6a8", url_path="alerts") -_indicators_page = st.Page(indicators.show, title="Indicators", icon="\U0001f321\ufe0f", url_path="indicators") -_advanced_page = st.Page(advanced.show, title="Advanced", icon="\U0001f52c", url_path="advanced") - -pages = { - "Dashboard": [ - _overview_page, - _single_analysis_page, - ], - "Compare": [ - _comparisons_page, - _reach_page, - ], - "Monitor": [ - _alerts_page, - _indicators_page, - _advanced_page, - ], -} - -render_main_nav() - -pg = st.navigation(pages) - -# Store page refs for cross-page navigation -st.session_state["_page_single_analysis"] = _single_analysis_page - -inventory_df = get_inventory() -site_count = len(inventory_df) if not inventory_df.empty else 0 -st.caption(f"{site_count} sites | {len(AVAILABLE_PLOTS)} plot types") - -pg.run() - -render_footer() +# Execute the shared shell (single source of truth in app.py) +import hydrology.app.app # noqa: F401,E402 diff --git a/hydrology/app/styles.py b/hydrology/app/styles.py index 6d41071..bbb3a5c 100644 --- a/hydrology/app/styles.py +++ b/hydrology/app/styles.py @@ -4,81 +4,238 @@ import streamlit as st from typing import Dict, Any, Optional +import numpy as np import pandas as pd from html import escape +def _inject_css(css: str) -> None: + """Inject CSS without dumping rules as visible page text. + + Streamlit sometimes sanitizes bare <style>/<link> in st.markdown and can + leave raw CSS visible. Prefer st.html / components.html (height=0). + """ + payload = f"<style>\n{css}\n</style>" + # Streamlit >= 1.33 + if hasattr(st, "html"): + try: + st.html(payload) + return + except Exception: + pass + try: + import streamlit.components.v1 as components + + components.html(payload, height=0, scrolling=False) + return + except Exception: + pass + # Last resort — may still work on older hosts + st.markdown(payload, unsafe_allow_html=True) + + def apply_custom_css(): - """Apply custom CSS for a polished dark theme.""" - st.markdown(""" - <style> + """Apply custom CSS for a refined dark theme with premium fluid motion. + + Design language: clean, sleek, detailed — dense where useful, airy where + maps/plots need space. Motion is short and deliberate so Streamlit reruns + stay snappy (not gimmicky loops). + + Motion (subtle, performant, replay-safe under Streamlit reruns): + - Short fade + lift entrances on cards/panels. + - Hover lifts + accent intensification on actionable surfaces. + - Button transitions + active feedback. + - Status chip polish. + - Respects prefers-reduced-motion. + + Inspiration: elevated animated "award-winning" web UI patterns (see + https://x.com/twetsfyp/status/2065283731833651709 for the reference video + of modern fluid website design that informed the polish direction). + """ + css = """ + @import url("https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap"); + :root { - --hydro-bg: #08111f; - --hydro-panel: #101c2e; - --hydro-panel-2: #13283d; - --hydro-border: rgba(118, 169, 192, 0.26); - --hydro-text: #e7eef7; - --hydro-muted: #8da2b8; - --hydro-accent: #4ecdc4; - --hydro-accent-2: #78a6ff; - --hydro-warn: #fbbf24; + --hydro-bg: #070d16; + --hydro-panel: #0c1624; + --hydro-panel-2: #101f33; + --hydro-border: rgba(125, 170, 200, 0.18); + --hydro-text: #edf3fa; + --hydro-muted: #8b9cb0; + --hydro-accent: #3dd6c6; + --hydro-accent-2: #6b9fff; + --hydro-warn: #f0b429; + --hydro-danger: #f07178; + --hydro-radius: 12px; + --hydro-radius-sm: 8px; + --hydro-shadow: 0 12px 40px rgba(0, 0, 0, 0.35); + --hydro-font: "Inter", ui-sans-serif, system-ui, -apple-system, "Segoe UI", sans-serif; + --hydro-mono: "JetBrains Mono", ui-monospace, "SF Mono", Consolas, monospace; + } + + /* Premium fluid motion keyframes — award-winning feel, short durations for snappy Streamlit. */ + @keyframes fadeInUp { + from { opacity: 0; transform: translateY(8px); } + to { opacity: 1; transform: translateY(0); } + } + @keyframes cardPop { + from { opacity: 0; transform: translateY(6px) scale(0.99); } + to { opacity: 1; transform: translateY(0) scale(1); } + } + @keyframes subtlePulse { + 0%, 100% { box-shadow: 0 0 0 0 rgba(61, 214, 198, 0.0); } + 50% { box-shadow: 0 0 0 6px rgba(61, 214, 198, 0.08); } + } + @keyframes shimmer { + 0% { transform: translateX(-120%); } + 100% { transform: translateX(220%); } + } + @keyframes accentShift { + 0%, 100% { border-color: rgba(125, 170, 200, 0.18); } + 50% { border-color: rgba(61, 214, 198, 0.35); } + } + + html, body, [class*="css"] { + font-family: var(--hydro-font); } .stApp { background: - radial-gradient(circle at 16% 8%, rgba(78, 205, 196, 0.10), transparent 26rem), - radial-gradient(circle at 90% 12%, rgba(120, 166, 255, 0.10), transparent 30rem), - linear-gradient(180deg, #08111f 0%, #0d1420 48%, #090f18 100%); + radial-gradient(ellipse 80% 50% at 10% -10%, rgba(61, 214, 198, 0.09), transparent 50%), + radial-gradient(ellipse 60% 40% at 100% 0%, rgba(107, 159, 255, 0.08), transparent 45%), + radial-gradient(ellipse 50% 30% at 50% 100%, rgba(61, 214, 198, 0.04), transparent 50%), + linear-gradient(180deg, #070d16 0%, #0a121e 55%, #060b13 100%); color: var(--hydro-text); + letter-spacing: -0.01em; + } + + /* Hide Streamlit chrome that fights the product shell */ + #MainMenu { visibility: hidden; } + header[data-testid="stHeader"] { + background: transparent; + border: none; } + div[data-testid="stToolbar"] { display: none; } + footer { visibility: hidden; } + .stDeployButton { display: none; } - /* Main container padding. Extra top clearance keeps Streamlit's deploy ribbon - from covering the dashboard header on hosted and preview builds. */ + /* Scrollbar */ + ::-webkit-scrollbar { width: 8px; height: 8px; } + ::-webkit-scrollbar-track { background: transparent; } + ::-webkit-scrollbar-thumb { + background: rgba(125, 170, 200, 0.28); + border-radius: 999px; + } + ::-webkit-scrollbar-thumb:hover { background: rgba(61, 214, 198, 0.45); } + + /* Main container — top clearance for hosted deploy ribbon / toolbar ghost */ .main .block-container { - padding-top: 4.25rem; - padding-bottom: 2rem; - max-width: 1480px; + padding-top: 3.5rem; + padding-bottom: 2.5rem; + padding-left: 1.5rem; + padding-right: 1.5rem; + max-width: 1440px; + } + + h1, h2, h3, h4 { + font-family: var(--hydro-font); + letter-spacing: -0.02em; + font-weight: 600; + color: var(--hydro-text); + } + + p, label, .stMarkdown { + color: var(--hydro-text); + } + + code, pre, .stCode { + font-family: var(--hydro-mono) !important; } /* Card-style containers */ .metric-card { - background: linear-gradient(135deg, rgba(30, 58, 95, 0.95) 0%, rgba(13, 27, 42, 0.95) 100%); - border-radius: 8px; - padding: 1rem; + background: linear-gradient(160deg, rgba(16, 31, 51, 0.95) 0%, rgba(10, 18, 30, 0.98) 100%); + border-radius: var(--hydro-radius); + padding: 1rem 1.1rem; border: 1px solid var(--hydro-border); margin-bottom: 0.5rem; - box-shadow: 0 14px 34px rgba(0, 0, 0, 0.22); + box-shadow: var(--hydro-shadow); + backdrop-filter: blur(8px); } .dashboard-hero { + position: relative; + overflow: hidden; border: 1px solid var(--hydro-border); - border-radius: 8px; - padding: 1rem 1.1rem; - margin: 0 0 1rem 0; - background: linear-gradient(135deg, rgba(16, 28, 46, 0.92), rgba(12, 40, 55, 0.72)); - box-shadow: 0 16px 38px rgba(0, 0, 0, 0.28); + border-radius: var(--hydro-radius); + padding: 1.15rem 1.35rem 1.2rem; + margin: 0 0 0.85rem 0; + background: + linear-gradient(135deg, rgba(12, 24, 40, 0.96), rgba(8, 28, 42, 0.88)); + box-shadow: var(--hydro-shadow); + animation: fadeInUp 0.28s cubic-bezier(0.2, 0.8, 0.2, 1) both; + will-change: transform; + } + + .dashboard-hero::before { + content: ""; + position: absolute; + inset: 0 auto 0 0; + width: 3px; + background: linear-gradient(180deg, var(--hydro-accent), var(--hydro-accent-2)); + border-radius: 3px 0 0 3px; } .dashboard-hero .eyebrow { color: var(--hydro-accent); - font-size: 0.72rem; - font-weight: 700; - letter-spacing: 0.08em; + font-size: 0.68rem; + font-weight: 600; + letter-spacing: 0.12em; text-transform: uppercase; - margin-bottom: 0.25rem; + margin-bottom: 0.35rem; + opacity: 0.95; } .dashboard-hero h1 { - font-size: 1.85rem; + font-size: clamp(1.45rem, 2.2vw, 1.85rem); line-height: 1.15; margin: 0; color: var(--hydro-text); + font-weight: 700; + letter-spacing: -0.03em; } .dashboard-hero p { - margin: 0.4rem 0 0 0; + margin: 0.45rem 0 0 0; color: var(--hydro-muted); - max-width: 78ch; + max-width: 72ch; + font-size: 0.92rem; + line-height: 1.45; + font-weight: 400; + } + + .dashboard-meta { + display: flex; + flex-wrap: wrap; + gap: 0.4rem; + margin: 0.15rem 0 0.9rem; + } + + .dashboard-meta .meta-pill { + font-size: 0.72rem; + font-weight: 600; + letter-spacing: 0.02em; + color: var(--hydro-muted); + border: 1px solid var(--hydro-border); + background: rgba(255,255,255,0.03); + border-radius: 999px; + padding: 0.28rem 0.65rem; + font-family: var(--hydro-mono); + white-space: nowrap; + } + + .dashboard-meta .meta-pill + .meta-pill { + margin-left: 0; /* flex gap handles spacing; avoid sticky paste */ } .workflow-strip { @@ -112,23 +269,28 @@ def apply_custom_css(): .workspace-panel { border: 1px solid var(--hydro-border); - border-radius: 8px; - background: linear-gradient(180deg, rgba(10, 21, 36, 0.92), rgba(7, 17, 29, 0.88)); - padding: 0.95rem; - box-shadow: 0 14px 34px rgba(0, 0, 0, 0.20); + border-radius: var(--hydro-radius); + background: linear-gradient(180deg, rgba(12, 22, 36, 0.94), rgba(8, 15, 26, 0.92)); + padding: 1rem 1.1rem; + box-shadow: 0 10px 28px rgba(0, 0, 0, 0.22); + animation: fadeInUp 0.22s ease-out both; + will-change: transform; + backdrop-filter: blur(6px); } .workspace-panel h3 { - margin: 0 0 0.25rem 0; - font-size: 1rem; + margin: 0 0 0.3rem 0; + font-size: 0.95rem; + font-weight: 600; color: var(--hydro-text); + letter-spacing: -0.015em; } .workspace-panel p { margin: 0; color: var(--hydro-muted); font-size: 0.82rem; - line-height: 1.35; + line-height: 1.4; } .status-chip-row { @@ -149,12 +311,17 @@ def apply_custom_css(): border: 1px solid rgba(118, 169, 192, 0.22); color: var(--hydro-text); background: rgba(255, 255, 255, 0.055); + transition: transform 0.12s ease, border-color 0.12s ease, background 0.12s ease; } .status-chip.ready { border-color: rgba(78, 205, 196, 0.48); } + .status-chip:hover { + transform: scale(1.02); + } + .status-chip.limited { border-color: rgba(255, 193, 7, 0.55); } @@ -172,18 +339,26 @@ def apply_custom_css(): .action-card { display: block; - border: 1px solid rgba(118, 169, 192, 0.20); - border-radius: 8px; - padding: 0.75rem; - background: rgba(12, 21, 34, 0.72); + border: 1px solid rgba(125, 170, 200, 0.16); + border-radius: var(--hydro-radius); + padding: 0.85rem 0.9rem; + background: rgba(10, 18, 30, 0.82); color: var(--hydro-text); text-decoration: none; - min-height: 92px; + min-height: 96px; + transition: transform 0.18s cubic-bezier(0.2, 0.8, 0.2, 1), + box-shadow 0.18s ease, + border-color 0.18s ease, + background 0.18s ease; + animation: cardPop 0.24s cubic-bezier(0.2, 0.8, 0.2, 1) both; + will-change: transform; } .action-card:hover { - border-color: rgba(78, 205, 196, 0.62); - background: rgba(78, 205, 196, 0.11); + border-color: rgba(61, 214, 198, 0.5); + background: rgba(61, 214, 198, 0.08); + transform: translateY(-2px); + box-shadow: 0 14px 32px rgba(0, 0, 0, 0.32); text-decoration: none; color: var(--hydro-text); } @@ -210,16 +385,29 @@ def apply_custom_css(): } .plot-card { - border: 1px solid rgba(118, 169, 192, 0.18); - border-radius: 8px; - padding: 0.72rem 0.78rem; - background: rgba(12, 21, 34, 0.76); + position: relative; + border: 1px solid rgba(125, 170, 200, 0.14); + border-radius: var(--hydro-radius-sm); + padding: 0.78rem 0.85rem; + background: rgba(10, 18, 30, 0.88); min-height: 112px; - box-shadow: inset 0 1px 0 rgba(255,255,255,0.03); + box-shadow: inset 0 1px 0 rgba(255,255,255,0.04); + transition: transform 0.16s cubic-bezier(0.2, 0, 0, 1), + box-shadow 0.16s ease, + border-color 0.16s ease; + animation: cardPop 0.22s ease-out both; + will-change: transform; + cursor: help; } .plot-card.ready { - border-color: rgba(78, 205, 196, 0.35); + border-color: rgba(61, 214, 198, 0.32); + } + + .plot-card:hover { + transform: translateY(-2px); + box-shadow: 0 10px 24px rgba(0, 0, 0, 0.28), inset 0 1px 0 rgba(255,255,255,0.04); + z-index: 20; } .plot-card.limited { @@ -267,6 +455,74 @@ def apply_custom_css(): color: #f87171; } + .plot-card .tile-info, + .insight-card .tile-info { + position: absolute; + top: 0.45rem; + right: 0.5rem; + width: 1.05rem; + height: 1.05rem; + border-radius: 999px; + border: 1px solid rgba(78, 205, 196, 0.45); + color: var(--hydro-accent); + font-size: 0.62rem; + font-weight: 800; + line-height: 1.05rem; + text-align: center; + opacity: 0.75; + pointer-events: none; + } + + .tile-tip { + display: none; + position: absolute; + left: 0.35rem; + right: 0.35rem; + bottom: calc(100% + 8px); + z-index: 40; + padding: 0.65rem 0.75rem; + border-radius: 10px; + background: rgba(8, 16, 28, 0.97); + border: 1px solid rgba(78, 205, 196, 0.35); + box-shadow: 0 14px 32px rgba(0, 0, 0, 0.45); + color: #e7eef7; + font-size: 0.74rem; + line-height: 1.35; + white-space: normal; + pointer-events: none; + max-height: min(42vh, 220px); + overflow: auto; + -webkit-overflow-scrolling: touch; + } + + .tile-tip::after { + content: ""; + position: absolute; + left: 1.1rem; + top: 100%; + border: 6px solid transparent; + border-top-color: rgba(78, 205, 196, 0.35); + } + + /* Desktop hover + keyboard/touch focus (tap tile once on mobile) */ + .plot-card:hover .tile-tip, + .insight-card:hover .tile-tip, + .plot-card:focus .tile-tip, + .insight-card:focus .tile-tip, + .plot-card:focus-within .tile-tip, + .insight-card:focus-within .tile-tip, + .plot-card:active .tile-tip, + .insight-card:active .tile-tip { + display: block; + animation: fadeInUp 0.12s ease-out both; + } + + .plot-card:focus, + .insight-card:focus { + outline: 2px solid rgba(78, 205, 196, 0.55); + outline-offset: 2px; + } + .insight-board { display: grid; grid-template-columns: repeat(4, minmax(0, 1fr)); @@ -275,15 +531,44 @@ def apply_custom_css(): } .insight-card { - background: rgba(15, 27, 44, 0.86); - border: 1px solid rgba(118, 169, 192, 0.20); - border-radius: 8px; - padding: 0.8rem; + position: relative; + background: rgba(12, 22, 36, 0.92); + border: 1px solid rgba(125, 170, 200, 0.14); + border-radius: var(--hydro-radius); + padding: 0.9rem; min-height: 128px; + transition: transform 0.16s cubic-bezier(0.2, 0, 0, 1), + box-shadow 0.16s ease, + border-color 0.16s ease; + animation: fadeInUp 0.22s ease-out both; + will-change: transform; + cursor: help; } .insight-card.ready { - border-top: 3px solid var(--hydro-accent); + border-top: 2px solid var(--hydro-accent); + } + + .insight-card:hover { + transform: translateY(-2px); + box-shadow: 0 12px 28px rgba(0, 0, 0, 0.26); + z-index: 20; + } + + .insight-card .tile-info { + position: absolute; + top: 0.5rem; + right: 0.55rem; + width: 1.05rem; + height: 1.05rem; + border-radius: 999px; + border: 1px solid rgba(78, 205, 196, 0.45); + color: var(--hydro-accent); + font-size: 0.62rem; + font-weight: 800; + line-height: 1.05rem; + text-align: center; + opacity: 0.75; } .insight-card.limited { @@ -319,27 +604,29 @@ def apply_custom_css(): /* Site header styling */ .site-header { - background: linear-gradient(90deg, rgba(78, 205, 196, 0.12), rgba(120, 166, 255, 0.06), transparent); - padding: 0.85rem 1rem; + background: linear-gradient(100deg, rgba(61, 214, 198, 0.10), rgba(107, 159, 255, 0.05), transparent 70%); + padding: 0.95rem 1.1rem; margin-bottom: 1rem; - border-left: 4px solid var(--hydro-accent); - border-top: 1px solid rgba(78, 205, 196, 0.18); - border-bottom: 1px solid rgba(78, 205, 196, 0.10); - border-radius: 8px; + border-left: 3px solid var(--hydro-accent); + border: 1px solid rgba(61, 214, 198, 0.14); + border-left-width: 3px; + border-radius: var(--hydro-radius); + animation: fadeInUp 0.2s ease-out both; } .site-header h1 { margin: 0; color: var(--hydro-text); - font-size: 1.42rem; + font-size: 1.35rem; font-weight: 600; - letter-spacing: 0; + letter-spacing: -0.02em; } .site-header p { - margin: 0.25rem 0 0 0; + margin: 0.3rem 0 0 0; color: var(--hydro-muted); - font-size: 0.85rem; + font-size: 0.82rem; + font-family: var(--hydro-mono); } /* Data availability badges */ @@ -381,22 +668,28 @@ def apply_custom_css(): /* Metric styling */ [data-testid="stMetricValue"] { - font-size: 1.75rem; + font-size: 1.55rem; + font-weight: 700; + letter-spacing: -0.02em; color: var(--hydro-text); + font-variant-numeric: tabular-nums; } [data-testid="stMetricLabel"] { - font-size: 0.85rem; + font-size: 0.78rem; + font-weight: 600; + letter-spacing: 0.04em; + text-transform: uppercase; color: var(--hydro-muted); } [data-testid="stMetric"] { - background: rgba(16, 28, 46, 0.72); - border: 1px solid rgba(118, 169, 192, 0.18); - border-radius: 8px; - padding: 0.75rem 0.85rem; - min-height: 96px; - box-shadow: inset 0 1px 0 rgba(255,255,255,0.03); + background: rgba(12, 22, 36, 0.88); + border: 1px solid rgba(125, 170, 200, 0.14); + border-radius: var(--hydro-radius); + padding: 0.85rem 0.95rem; + min-height: 92px; + box-shadow: inset 0 1px 0 rgba(255,255,255,0.04); } /* Sidebar is hidden so the app works from the main workspace instead of @@ -409,36 +702,113 @@ def apply_custom_css(): .main-nav { display: flex; flex-wrap: wrap; - gap: 0.45rem; + gap: 0.35rem; align-items: center; - margin: 0.25rem 0 1rem 0; - padding: 0.55rem; + margin: 0 0 1rem 0; + padding: 0.4rem; border: 1px solid var(--hydro-border); - border-radius: 8px; - background: rgba(255, 255, 255, 0.035); + border-radius: 999px; + background: rgba(8, 14, 24, 0.72); + backdrop-filter: blur(10px); + box-shadow: 0 8px 24px rgba(0,0,0,0.2); + animation: fadeInUp 0.2s ease-out both; } .main-nav a { display: inline-flex; align-items: center; gap: 0.35rem; - padding: 0.42rem 0.7rem; - border-radius: 7px; - border: 1px solid rgba(118, 169, 192, 0.18); - color: var(--hydro-text); + padding: 0.48rem 0.9rem; + border-radius: 999px; + border: 1px solid transparent; + color: var(--hydro-muted); text-decoration: none; - font-size: 0.86rem; + font-size: 0.84rem; font-weight: 600; - background: rgba(8, 18, 32, 0.55); + background: transparent; + transition: transform 0.14s ease, border-color 0.14s ease, background 0.14s ease, color 0.14s ease; } .main-nav a:hover { - border-color: rgba(78, 205, 196, 0.65); - background: rgba(78, 205, 196, 0.12); + border-color: rgba(61, 214, 198, 0.35); + background: rgba(61, 214, 198, 0.10); + transform: translateY(-1px); color: var(--hydro-text); text-decoration: none; } + .main-nav a.active { + color: var(--hydro-bg); + background: linear-gradient(135deg, var(--hydro-accent), #2bb8ab); + border-color: transparent; + box-shadow: 0 4px 14px rgba(61, 214, 198, 0.25); + } + + .main-nav a:active { + transform: translateY(0) scale(0.985); + } + + /* Streamlit widgets — refined controls */ + div[data-baseweb="select"] > div, + div[data-baseweb="input"] > div, + .stTextInput input, + .stNumberInput input, + .stDateInput input { + background-color: rgba(8, 14, 24, 0.9) !important; + border-color: rgba(125, 170, 200, 0.22) !important; + border-radius: var(--hydro-radius-sm) !important; + color: var(--hydro-text) !important; + } + + .stSelectbox label, .stMultiSelect label, .stTextInput label, + .stNumberInput label, .stDateInput label, .stSlider label { + font-size: 0.78rem !important; + font-weight: 600 !important; + letter-spacing: 0.03em !important; + color: var(--hydro-muted) !important; + } + + .stTabs [data-baseweb="tab-list"] { + gap: 0.25rem; + background: rgba(8, 14, 24, 0.55); + border-radius: 999px; + padding: 0.3rem; + border: 1px solid var(--hydro-border); + } + + .stTabs [data-baseweb="tab"] { + border-radius: 999px; + padding: 0.4rem 0.9rem; + color: var(--hydro-muted); + font-weight: 600; + font-size: 0.84rem; + } + + .stTabs [aria-selected="true"] { + background: rgba(61, 214, 198, 0.14) !important; + color: var(--hydro-text) !important; + } + + .stExpander { + border: 1px solid var(--hydro-border) !important; + border-radius: var(--hydro-radius) !important; + background: rgba(10, 18, 30, 0.55) !important; + } + + div[data-testid="stAlert"] { + border-radius: var(--hydro-radius-sm); + border: 1px solid var(--hydro-border); + background: rgba(12, 22, 36, 0.9); + } + + .stProgress > div > div { + background: linear-gradient(90deg, var(--hydro-accent), var(--hydro-accent-2)) !important; + } + + .stSpinner > div { + border-top-color: var(--hydro-accent) !important; + } + /* [data-testid="stSidebar"] { background: linear-gradient(180deg, #0a1524 0%, #07111d 100%); @@ -467,51 +837,115 @@ def apply_custom_css(): } */ - /* Button styling */ + /* Button styling — refined CTAs (subtlePulse kept for polish, not circus) */ .stButton > button { - border-radius: 8px; - font-weight: 500; - transition: all 0.2s ease; - border-color: rgba(78, 205, 196, 0.45); + border-radius: 999px; + font-weight: 600; + font-size: 0.88rem; + letter-spacing: -0.01em; + transition: transform 0.15s cubic-bezier(0.2, 0, 0, 1), + box-shadow 0.15s ease, + border-color 0.15s ease, + background 0.15s ease; + border-color: rgba(61, 214, 198, 0.35) !important; + background: rgba(10, 18, 30, 0.9) !important; + color: var(--hydro-text) !important; + min-height: 2.5rem; } .stButton > button:hover { - transform: translateY(-1px); - box-shadow: 0 4px 12px rgba(0,0,0,0.3); + transform: translateY(-2px); + box-shadow: 0 8px 20px rgba(0,0,0,0.32); + border-color: rgba(61, 214, 198, 0.7) !important; + } + + .stButton > button[kind="primary"], + .stButton > button[type="primary"], + .stButton > button[data-testid="baseButton-primary"] { + background: linear-gradient(135deg, #3dd6c6 0%, #2bb8ab 55%, #2499d6 100%) !important; + color: #061018 !important; + border: none !important; + box-shadow: 0 6px 18px rgba(61, 214, 198, 0.22); + animation: subtlePulse 2.8s ease-in-out infinite, fadeInUp 0.25s ease-out; + position: relative; + overflow: hidden; } - /* Plot container */ + .stButton > button[kind="primary"]::after, + .stButton > button[type="primary"]::after, + .stButton > button[data-testid="baseButton-primary"]::after { + content: ''; + position: absolute; + top: 0; + left: 0; + width: 40%; + height: 100%; + background: linear-gradient(90deg, transparent, rgba(255,255,255,0.22), transparent); + animation: shimmer 2.4s ease-in-out infinite; + pointer-events: none; + } + + .stButton > button[kind="primary"]:hover, + .stButton > button[type="primary"]:hover, + .stButton > button[data-testid="baseButton-primary"]:hover { + box-shadow: 0 10px 24px rgba(61, 214, 198, 0.32); + transform: translateY(-2px); + filter: brightness(1.04); + } + + .stButton > button:active { + transform: translateY(0) scale(0.98); + } + + .stButton > button:focus-visible { + outline: 2px solid rgba(61, 214, 198, 0.55); + outline-offset: 2px; + } + + /* Plot / map surfaces */ .plot-container { - background: rgba(14, 17, 23, 0.82); - border: 1px solid rgba(118, 169, 192, 0.18); - border-radius: 8px; + background: rgba(8, 14, 24, 0.88); + border: 1px solid rgba(125, 170, 200, 0.14); + border-radius: var(--hydro-radius); padding: 1rem; margin-bottom: 1rem; } div[data-testid="stPlotlyChart"], iframe[title="streamlit_folium.st_folium"] { - border-radius: 8px; + border-radius: var(--hydro-radius); overflow: hidden; + border: 1px solid rgba(125, 170, 200, 0.12); } div[data-testid="stDataFrame"] { - border: 1px solid rgba(118, 169, 192, 0.14); - border-radius: 8px; + border: 1px solid rgba(125, 170, 200, 0.12); + border-radius: var(--hydro-radius); overflow: hidden; } /* Footer styling */ .footer-info { text-align: center; - color: #6b7280; - font-size: 0.8rem; - padding: 1rem; - border-top: 1px solid rgba(118, 169, 192, 0.18); - margin-top: 2rem; + color: #6b7a8c; + font-size: 0.76rem; + padding: 1.25rem 1rem 0.5rem; + border-top: 1px solid rgba(125, 170, 200, 0.12); + margin-top: 2.25rem; + letter-spacing: 0.01em; } - /* Mobile responsiveness */ + .footer-info a { + color: var(--hydro-accent); + text-decoration: none; + font-weight: 600; + } + + .footer-info a:hover { + text-decoration: underline; + } + + /* Mobile / tablet responsiveness */ @media (max-width: 768px) { .main .block-container { padding-top: 3.5rem; @@ -519,16 +953,66 @@ def apply_custom_css(): padding-right: 0.5rem; } + /* Stack Streamlit columns so rating workshop + metrics don't squeeze */ + [data-testid="column"] { + min-width: 100% !important; + width: 100% !important; + flex: 1 1 100% !important; + } + + [data-testid="stHorizontalBlock"] { + flex-wrap: wrap !important; + gap: 0.35rem 0 !important; + } + .workflow-strip { grid-template-columns: 1fr 1fr; } .plot-board { grid-template-columns: 1fr 1fr; + gap: 0.55rem; } .insight-board { grid-template-columns: 1fr 1fr; + gap: 0.55rem; + } + + .plot-card, + .insight-card { + min-height: 0; + padding: 0.72rem 0.78rem; + padding-right: 1.7rem; + /* Avoid hover-lift stealing space / clipping tips on touch */ + transform: none !important; + } + + .plot-card .tile-info, + .insight-card .tile-info { + width: 1.35rem; + height: 1.35rem; + line-height: 1.35rem; + font-size: 0.72rem; + opacity: 0.95; + } + + /* Tips open below the tile on mobile so they aren't under the app chrome */ + .tile-tip { + bottom: auto; + top: calc(100% + 6px); + left: 0; + right: 0; + font-size: 0.78rem; + max-height: min(50vh, 260px); + box-shadow: 0 10px 28px rgba(0, 0, 0, 0.5); + } + + .tile-tip::after { + top: auto; + bottom: 100%; + border-top-color: transparent; + border-bottom-color: rgba(78, 205, 196, 0.35); } .dashboard-hero h1 { @@ -536,13 +1020,18 @@ def apply_custom_css(): } [data-testid="stMetricValue"] { - font-size: 1.2rem; + font-size: 1.15rem; } [data-testid="stMetricLabel"] { font-size: 0.75rem; } + /* Plotly: keep charts usable on narrow viewports */ + div[data-testid="stPlotlyChart"] { + min-height: 280px; + } + .site-header h1 { font-size: 1.1rem; } @@ -555,13 +1044,18 @@ def apply_custom_css(): font-size: 0.7rem; padding: 0.15rem 0.5rem; } - } - @media (max-width: 480px) { - [data-testid="column"] { - min-width: 100% !important; + .action-card-grid { + grid-template-columns: 1fr !important; } + .main-nav { + flex-wrap: wrap; + gap: 0.35rem; + } + } + + @media (max-width: 480px) { [data-testid="stMetricValue"] { font-size: 1rem; } @@ -582,76 +1076,130 @@ def apply_custom_css(): .insight-board { grid-template-columns: 1fr; } + + /* Tighter Plotly margins feel on very small phones */ + div[data-testid="stPlotlyChart"] { + min-height: 260px; + } + } + + /* Touch devices: prefer tap/focus over hover for tips */ + @media (hover: none) and (pointer: coarse) { + .plot-card, + .insight-card { + cursor: pointer; + -webkit-tap-highlight-color: rgba(78, 205, 196, 0.15); + } + + .plot-card:hover .tile-tip, + .insight-card:hover .tile-tip { + display: none; + } + + .plot-card:focus .tile-tip, + .insight-card:focus .tile-tip, + .plot-card:focus-within .tile-tip, + .insight-card:focus-within .tile-tip { + display: block; + } + } + + /* Respect user motion preference and add subtle global polish */ + @media (prefers-reduced-motion: reduce) { + .dashboard-hero, + .workspace-panel, + .action-card, + .plot-card, + .insight-card, + .status-chip, + .main-nav a, + .stButton > button, + .tile-tip { + animation: none !important; + transition: none !important; + transform: none !important; + } } - </style> - """, unsafe_allow_html=True) + + /* Extra container polish for Plotly/folium (map & chart surfaces feel premium) */ + div[data-testid="stPlotlyChart"], + iframe[title="streamlit_folium.st_folium"] { + transition: box-shadow 0.2s ease, border-color 0.2s ease; + } + + @media (hover: hover) { + div[data-testid="stPlotlyChart"]:hover, + iframe[title="streamlit_folium.st_folium"]:hover { + box-shadow: 0 8px 24px rgba(0, 0, 0, 0.22); + } + } + """ + _inject_css(css) def render_dashboard_hero(title: str, subtitle: str): """Render a compact first-screen dashboard header.""" st.markdown(f""" <div class="dashboard-hero"> - <div class="eyebrow">Pacific Northwest hydrology workspace</div> - <h1>{title}</h1> - <p>{subtitle}</p> + <div class="eyebrow">HydroPlot · Pacific Northwest</div> + <h1>{escape(str(title))}</h1> + <p>{escape(str(subtitle))}</p> </div> """, unsafe_allow_html=True) -def render_main_nav(): - """Render main-page navigation so the sidebar is not the primary workflow.""" - st.markdown(""" - <nav class="main-nav" aria-label="Main workflow navigation"> - <a href="overview" target="_self">Stations</a> - <a href="single-analysis" target="_self">Site Analysis</a> - <a href="comparisons" target="_self">Compare Sites</a> - <a href="reach-analysis" target="_self">Reach Tools</a> - <a href="alerts" target="_self">Current Check</a> - <a href="indicators" target="_self">Climate Indicators</a> - <a href="advanced" target="_self">More Tools</a> - </nav> - """, unsafe_allow_html=True) +def render_dashboard_meta(site_count: int = 0, plot_count: int = 0): + """Compact inventory strip under the hero (snappier than sidebar captions).""" + bits = [] + if site_count: + bits.append(f'<span class="meta-pill">{int(site_count):,} gages</span>') + if plot_count: + bits.append(f'<span class="meta-pill">{int(plot_count)} plot types</span>') + bits.append('<span class="meta-pill">USGS · Meteostat · NLDI</span>') + st.markdown( + '<div class="dashboard-meta">' + "".join(bits) + "</div>", + unsafe_allow_html=True, + ) -def render_workflow_strip(): - """Render visible navigation intent without replacing Streamlit navigation.""" - render_action_cards([ - { - "title": "Stations", - "href": "overview", - "body": "Find gages, inspect map layers, live flow context, and record coverage.", - }, - { - "title": "Site Analysis", - "href": "single-analysis", - "body": "Run guided or fully custom plot sets for one selected gage.", - }, - { - "title": "Compare Sites", - "href": "comparisons", - "body": "Check overlap and contrast records before heavier multi-site work.", - }, - { - "title": "Reach Tools", - "href": "reach-analysis", - "body": "Evaluate paired gages, gain/loss patterns, and reach behavior.", - }, - { - "title": "Current Check", - "href": "alerts", - "body": "Run manual threshold checks against latest available readings.", - }, - { - "title": "Climate Indicators", - "href": "indicators", - "body": "Review drought, SPI, precipitation, and climate-linked signals.", - }, - { - "title": "More Tools", - "href": "advanced", - "body": "Open specialized analysis utilities without cluttering core workflows.", - }, - ]) +def _nav_active_path() -> str: + """Best-effort active page path for pill highlight.""" + try: + path = (getattr(st, "context", None) and getattr(st.context, "url_path", None)) or "" + path = str(path).strip("/") + if path: + return path.split("/")[-1] + except Exception: + pass + # Fallback: Streamlit multipage often leaves last path fragment in session + for key in ("_hydro_active_page", "page"): + if key in st.session_state and st.session_state[key]: + return str(st.session_state[key]).strip("/") + return "overview" + + +def render_main_nav(active: str | None = None): + """Render main-page navigation so the sidebar is not the primary workflow.""" + active = (active or _nav_active_path() or "overview").strip("/") + items = [ + ("overview", "Stations"), + ("single-analysis", "Site Analysis"), + ("comparisons", "Compare Sites"), + ("reach-analysis", "Reach Analysis"), + ("watershed", "Watershed"), + ] + links = [] + for path, label in items: + cls = "active" if active == path or active.endswith(path) else "" + links.append( + f'<a class="{cls}" href="{path}" target="_self">{escape(label)}</a>' + ) + st.markdown( + '<nav class="main-nav" aria-label="Main workflow navigation">' + + "".join(links) + + "</nav>", + unsafe_allow_html=True, + ) def render_workspace_panel(title: str, body: str, chips: list[dict] | None = None): @@ -695,19 +1243,33 @@ def render_action_cards(cards: list[dict]): st.markdown('<div class="action-card-grid">' + "".join(html) + "</div>", unsafe_allow_html=True) +def _tile_tip_html(help_text: str | None) -> str: + """Optional hover/focus popup markup for HTML tiles (desktop + mobile tap).""" + if not help_text: + return "" + tip = escape(str(help_text)) + return ( + f'<span class="tile-info" aria-hidden="true">i</span>' + f'<div class="tile-tip" role="tooltip">{tip}</div>' + ) + + def render_plot_capability_board(cards: list[dict]): - """Render compact plot capability cards.""" + """Render compact plot capability cards with optional hover help.""" if not cards: return html_cards = [] for card in cards: - state = card.get("state", "ready") - title = card.get("title", "Plot") - body = card.get("body", "") - status = card.get("status", "Ready") + state = escape(str(card.get("state", "ready"))) + title = escape(str(card.get("title", "Plot"))) + body = escape(str(card.get("body", ""))) + status = escape(str(card.get("status", "Ready"))) + tip = _tile_tip_html(card.get("help") or card.get("tip")) html_cards.append( - f'<div class="plot-card {state}">' + f'<div class="plot-card {state}" tabindex="0" role="group" ' + f'aria-label="{title}">' + f"{tip}" f"<strong>{title}</strong>" f"<span>{body}</span>" f'<div class="status">{status}</div>' @@ -718,24 +1280,38 @@ def render_plot_capability_board(cards: list[dict]): '<div class="plot-board">' + "".join(html_cards) + "</div>", unsafe_allow_html=True, ) + st.caption("Desktop: hover a tile · Mobile: tap a tile once for the tip.") def render_insight_board(cards): - """Render data interpretation cards.""" + """Render data interpretation cards with optional hover help.""" if not cards: return html_cards = [] for card in cards: - state = getattr(card, "state", "ready") - title = getattr(card, "title", "") - value = getattr(card, "value", "") - body = getattr(card, "body", "") + if isinstance(card, dict): + state = card.get("state", "ready") + title = card.get("title", "") + value = card.get("value", "") + body = card.get("body", "") + help_text = card.get("help") or card.get("tip") or "" + else: + state = getattr(card, "state", "ready") + title = getattr(card, "title", "") + value = getattr(card, "value", "") + body = getattr(card, "body", "") + help_text = getattr(card, "help", "") or getattr(card, "tip", "") or "" + + tip = _tile_tip_html(help_text) + safe_title = escape(str(title)) html_cards.append( - f'<div class="insight-card {state}">' - f'<div class="label">{title}</div>' - f'<div class="value">{value}</div>' - f'<div class="body">{body}</div>' + f'<div class="insight-card {escape(str(state))}" tabindex="0" ' + f'role="group" aria-label="{safe_title}">' + f"{tip}" + f'<div class="label">{safe_title}</div>' + f'<div class="value">{escape(str(value))}</div>' + f'<div class="body">{escape(str(body))}</div>' "</div>" ) @@ -789,33 +1365,126 @@ def render_availability_badges(has_discharge: bool, has_stage: bool, climate_inf st.markdown(' '.join(badges), unsafe_allow_html=True) -def render_metric_cards(df_q: pd.DataFrame, df_merged: pd.DataFrame = None, discharge_col: str = 'Discharge_cfs'): - """Render key statistics as metric cards.""" +def render_metric_cards( + df_q: pd.DataFrame, + df_merged: pd.DataFrame = None, + discharge_col: str = "Discharge_cfs", + section: str = "all", +): + """Render key statistics with site-specific help tooltips. + + section: + - ``record``: record length + data points (site header strip) + - ``duration``: mean/peak + Q10/Q50/Q90 (belongs with flow-duration) + - ``all``: both (legacy / overview) + """ if df_q is None or df_q.empty: return - col1, col2, col3, col4 = st.columns(4) + from hydrology.app.interpretation import ( + compute_hydrologic_profile, + dynamic_metric_relevance, + format_metric_relevance_markdown, + metric_help_text, + ) - # Record length - if hasattr(df_q.index, 'min') and hasattr(df_q.index, 'max'): - years = (df_q.index.max() - df_q.index.min()).days / 365.25 - with col1: - st.metric("Record Length", f"{years:.1f} yrs", help="Total years of data") + def _help(key: str) -> str: + return metric_help_text(key, df_q, df_merged, discharge_col) - # Data points - with col2: - st.metric("Data Points", f"{len(df_q):,}", help="Number of daily observations") + show_record = section in ("all", "record") + show_duration = section in ("all", "duration") + profile = ( + compute_hydrologic_profile(df_q, df_merged, discharge_col) + if show_duration or section == "all" + else {} + ) - # Mean discharge - if discharge_col in df_q.columns: - mean_q = df_q[discharge_col].mean() + if show_record: + if section == "record": + st.caption("Record coverage for the selected period (hover metrics for tips).") + col1, col2 = st.columns(2) + if hasattr(df_q.index, "min") and hasattr(df_q.index, "max"): + years = (df_q.index.max() - df_q.index.min()).days / 365.25 + with col1: + st.metric("Record Length", f"{years:.1f} yrs", help=_help("record_length")) + with col2: + st.metric("Data Points", f"{len(df_q):,}", help=_help("data_points")) + + if show_duration and discharge_col in df_q.columns: + series = pd.to_numeric(df_q[discharge_col], errors="coerce").dropna() + series = series[series >= 0] + if len(series) == 0: + return + + if section == "duration": + st.markdown("##### Duration statistics") + st.caption( + "These map to the flow-duration curve below: " + "**Q10** ≈ high-flow end · **Q50** median · **Q90** low-flow end. " + "Hover each metric for site-specific context." + ) + + mean_q = float(series.mean()) + max_q = float(series.max()) + median_q = float(series.median()) + col3, col4 = st.columns(2) with col3: - st.metric("Mean Flow", f"{mean_q:,.0f} cfs", help="Average daily discharge") - - # Max discharge - max_q = df_q[discharge_col].max() + delta = None + if np.isfinite(mean_q) and np.isfinite(median_q) and median_q > 0: + skew_pct = (mean_q / median_q - 1.0) * 100 + if abs(skew_pct) >= 15: + delta = f"{skew_pct:+.0f}% vs median" + st.metric( + "Mean Flow", + f"{mean_q:,.0f} cfs", + delta=delta, + delta_color="off", + help=_help("mean_flow"), + ) with col4: - st.metric("Peak Flow", f"{max_q:,.0f} cfs", help="Maximum recorded discharge") + st.metric( + "Peak Flow", + f"{max_q:,.0f} cfs", + help=_help("peak_flow"), + ) + + if len(series) >= 30: + q10 = float(np.percentile(series, 90)) + q50 = float(np.percentile(series, 50)) + q90 = float(np.percentile(series, 10)) + c1, c2, c3 = st.columns(3) + with c1: + st.metric("Q10 (high)", f"{q10:,.0f} cfs", help=_help("q10")) + with c2: + st.metric("Q50 (median)", f"{q50:,.0f} cfs", help=_help("q50")) + with c3: + st.metric("Q90 (low)", f"{q90:,.0f} cfs", help=_help("q90")) + + card_keys = ["mean_flow", "peak_flow", "q10", "q50", "q90"] + if section == "all": + card_keys = ["record_length", "data_points"] + card_keys + regime_label = ( + profile.get("regime_plain", "this period") if profile.get("ok") else "this period" + ) + expander_title = f"What these duration stats mean — {regime_label}" + with st.expander(expander_title, expanded=False): + if profile.get("ok"): + st.caption( + f"Site-specific: Q10/Q90≈{profile['q10_q90']:.1f}, " + f"regime **{regime_label}**." + ) + rows = dynamic_metric_relevance( + df_q, df_merged, discharge_col=discharge_col, metric_keys=card_keys + ) + if rows: + st.dataframe(pd.DataFrame(rows), width="stretch", hide_index=True) + st.markdown( + format_metric_relevance_markdown( + ["spi_sri", "rating_r2"], + df_q=df_q, + df_merged=df_merged, + ) + ) def render_progress_bar(current: int, total: int, text: str = "Loading..."): @@ -828,7 +1497,8 @@ def render_footer(): """Render footer with app info.""" st.markdown(""" <div class="footer-info"> - <p>Hydrology Analysis Dashboard | Data: USGS NWIS & Meteostat | - <a href="https://github.com/abstractionisms/Hydroanalysispy" target="_blank">GitHub</a></p> + <p><strong>HydroPlot</strong> · USGS NWIS · Meteostat · NLDI · NWM + · <a href="https://github.com/abstractionisms/Hydroanalysispy" target="_blank">GitHub</a> + · Built for clean gage-to-reach workflows</p> </div> """, unsafe_allow_html=True) diff --git a/hydrology/data/__init__.py b/hydrology/data/__init__.py index eb3edbc..069bdfb 100644 --- a/hydrology/data/__init__.py +++ b/hydrology/data/__init__.py @@ -17,11 +17,15 @@ search_sites, ) from .nwm import NWMClient, NWMForecast, compare_nwm_usgs, get_forecast_skill +from .groundwater import ( + fetch_usgs_groundwater_measurements, + normalize_usgs_groundwater_measurements, +) # HyRiver imports are optional (require conda-forge packages) try: from .hyriver import ( - get_watershed_boundary, get_flowlines, get_basin_characteristics, + get_watershed_boundary, get_flowlines, get_navigation_flowlines, get_basin_characteristics, get_daymet_climate, get_nid_dams, get_elevation_profile, ) _HYRIVER_AVAILABLE = True @@ -46,9 +50,12 @@ 'NWMForecast', 'compare_nwm_usgs', 'get_forecast_skill', + 'fetch_usgs_groundwater_measurements', + 'normalize_usgs_groundwater_measurements', # HyRiver (optional) 'get_watershed_boundary', 'get_flowlines', + 'get_navigation_flowlines', 'get_basin_characteristics', 'get_daymet_climate', 'get_nid_dams', diff --git a/hydrology/data/climate.py b/hydrology/data/climate.py index ba96605..b7b9efb 100644 --- a/hydrology/data/climate.py +++ b/hydrology/data/climate.py @@ -1,12 +1,20 @@ """ -Climate data fetching using Meteostat for hydrology package. +Climate data fetching for the hydrology package. -Provides functions to fetch temperature and precipitation data -from the Meteostat API for hydrological analysis. +Providers: + - Open-Meteo Historical archive (no key, gridded, excellent modern coverage) + - Meteostat station daily (station-based; may pick dead nearest stations) + - (Daymet is optional via hyriver and needs NASA Earthdata) + +Meteostat caveat: Point(lat, lon) picks the *nearest* station even if that +station has no data in the requested window (e.g. Walla Walla 72788 ends 1988). +We therefore try several nearby stations with coverage overlap before failing. """ +from __future__ import annotations + +from typing import Optional, Dict, List, Any import pandas as pd -from typing import Optional, Tuple, Dict from ..core.logging_setup import get_logger from ..core.timezone import ensure_utc @@ -14,7 +22,6 @@ logger = get_logger(__name__) -# Lazy imports for optional dependencies def _get_meteostat(): try: from meteostat import Point, Daily, Stations @@ -23,107 +30,145 @@ def _get_meteostat(): return None, None, None +def _normalize_ts(value) -> pd.Timestamp: + ts = pd.Timestamp(value) + if ts.tzinfo is not None: + ts = ts.tz_convert("UTC").tz_localize(None) + return ts.normalize() + + +def _station_covers_window(station_row, start_date: pd.Timestamp, end_date: pd.Timestamp) -> bool: + """True if station daily inventory overlaps the requested window at all.""" + start = _normalize_ts(start_date) + end = _normalize_ts(end_date) + raw_start = station_row.get("daily_start", None) if hasattr(station_row, "get") else station_row["daily_start"] if "daily_start" in station_row.index else None + raw_end = station_row.get("daily_end", None) if hasattr(station_row, "get") else station_row["daily_end"] if "daily_end" in station_row.index else None + try: + if pd.isna(raw_start) or pd.isna(raw_end): + # Unknown inventory — still try fetching + return True + s0 = _normalize_ts(raw_start) + s1 = _normalize_ts(raw_end) + except Exception: + return True + # Overlap if station ends after request start and starts before request end + return s1 >= start and s0 <= end + + +def _process_meteostat_daily( + data: pd.DataFrame, + include_temp: bool, + include_precip: bool, +) -> Optional[pd.DataFrame]: + if data is None or data.empty: + return None + + cols_to_keep = [] + rename_map = {} + if include_temp and "tavg" in data.columns: + cols_to_keep.append("tavg") + rename_map["tavg"] = "Temp_C" + if include_precip and "prcp" in data.columns: + cols_to_keep.append("prcp") + rename_map["prcp"] = "Precip_mm" + if not cols_to_keep: + return None + + climate_df = data[cols_to_keep].copy().rename(columns=rename_map) + climate_df = ensure_utc(climate_df) + + if "Precip_mm" in climate_df.columns: + climate_df["Precip_mm"] = climate_df["Precip_mm"].fillna(0) + if "Temp_C" in climate_df.columns: + climate_df["Temp_C"] = climate_df["Temp_C"].ffill().bfill() + + # Reject nearly-empty series + usable = 0 + if "Precip_mm" in climate_df.columns: + usable = max(usable, int(climate_df["Precip_mm"].notna().sum())) + if "Temp_C" in climate_df.columns: + usable = max(usable, int(climate_df["Temp_C"].notna().sum())) + if usable < 30: + return None + return climate_df + + def fetch_climate_data( latitude: float, longitude: float, start_date: pd.Timestamp, end_date: pd.Timestamp, include_temp: bool = True, - include_precip: bool = True + include_precip: bool = True, ) -> Optional[pd.DataFrame]: """ - Fetch daily climate data using Meteostat. - - Fetches temperature and/or precipitation data for a location. - Fills missing precipitation with zeros and forwards/backwards fills - missing temperature values. - - Args: - latitude: Latitude in decimal degrees - longitude: Longitude in decimal degrees - start_date: Start datetime - end_date: End datetime - include_temp: Include temperature data (column: 'Temp_C') - include_precip: Include precipitation data (column: 'Precip_mm') - - Returns: - DataFrame with UTC datetime index and climate columns, or None if failed + Fetch daily climate data using Meteostat with multi-station fallback. - Example: - >>> climate = fetch_climate_data(47.6593, -117.4491, - ... pd.Timestamp('2020-01-01'), - ... pd.Timestamp('2023-12-31')) - >>> print(climate[['Temp_C', 'Precip_mm']].describe()) + Tries nearby stations that actually inventory-cover the request window, + instead of trusting Point() which may select a dead nearest station. """ - # Validate date order if start_date > end_date: - logger.warning(f"Start date ({start_date.date()}) after end date ({end_date.date()}) - swapping") + logger.warning( + f"Start date ({start_date.date()}) after end date ({end_date.date()}) - swapping" + ) start_date, end_date = end_date, start_date - logger.info(f"Fetching climate data: Lat={latitude:.4f}, Lon={longitude:.4f}, " - f"{start_date.date()} to {end_date.date()}") - - try: - Point, Daily, _ = _get_meteostat() - if Point is None or Daily is None: - logger.warning("meteostat not installed - skipping Meteostat source") - return None - - # Create location point and fetch data - location = Point(latitude, longitude) - data = Daily(location, start_date, end_date) - data = data.fetch() - - if data is None or data.empty: - logger.warning("Meteostat returned no data for this location/period") - return None - - # Select and rename columns - cols_to_keep = [] - rename_map = {} + logger.info( + f"Fetching climate data: Lat={latitude:.4f}, Lon={longitude:.4f}, " + f"{start_date.date()} to {end_date.date()}" + ) - if include_temp and 'tavg' in data.columns: - cols_to_keep.append('tavg') - rename_map['tavg'] = 'Temp_C' - elif include_temp: - logger.warning("Temperature requested but 'tavg' column not in Meteostat data") + Point, Daily, Stations = _get_meteostat() + if Point is None or Daily is None: + logger.warning("meteostat not installed - skipping Meteostat source") + return None - if include_precip and 'prcp' in data.columns: - cols_to_keep.append('prcp') - rename_map['prcp'] = 'Precip_mm' - elif include_precip: - logger.warning("Precipitation requested but 'prcp' column not in Meteostat data") + start_date = _normalize_ts(start_date) + end_date = _normalize_ts(end_date) - if not cols_to_keep: - logger.warning("No requested climate columns available in Meteostat data") - return None + try: + # 1) Multi-station search with coverage filter + if Stations is not None: + nearby = Stations().nearby(latitude, longitude).fetch(15) + if nearby is not None and not nearby.empty: + # Prefer stations with inventory overlap, ordered by distance + if "distance" in nearby.columns: + nearby = nearby.sort_values("distance") + candidates = [] + for sid, row in nearby.iterrows(): + if _station_covers_window(row, start_date, end_date): + candidates.append(sid) + # Also keep first few regardless as last-chance tries + for sid in list(nearby.index)[:5]: + if sid not in candidates: + candidates.append(sid) + + for sid in candidates[:10]: + try: + data = Daily(str(sid), start_date, end_date).fetch() + climate_df = _process_meteostat_daily( + data, include_temp, include_precip + ) + if climate_df is not None and not climate_df.empty: + name = nearby.loc[sid].get("name", sid) if sid in nearby.index else sid + logger.info( + f"Meteostat station {sid} ({name}): {len(climate_df)} rows" + ) + return climate_df + except Exception as e: + logger.debug(f"Meteostat station {sid} failed: {e}") + continue + + # 2) Legacy Point() interpolation as last Meteostat attempt + location = Point(latitude, longitude) + data = Daily(location, start_date, end_date).fetch() + climate_df = _process_meteostat_daily(data, include_temp, include_precip) + if climate_df is not None and not climate_df.empty: + logger.info(f"Climate data via Point(): {len(climate_df)} rows") + return climate_df - # Create climate dataframe - climate_df = data[cols_to_keep].copy() - climate_df = climate_df.rename(columns=rename_map) - - # Ensure UTC index - climate_df = ensure_utc(climate_df) - - # Fill missing values - if 'Precip_mm' in climate_df.columns: - # Fill missing precipitation with 0 (assume no rain on missing days) - missing_precip = climate_df['Precip_mm'].isna().sum() - climate_df['Precip_mm'] = climate_df['Precip_mm'].fillna(0) - if missing_precip > 0: - logger.info(f"Filled {missing_precip} missing precipitation values with 0") - - if 'Temp_C' in climate_df.columns: - # Forward/backward fill missing temperature - missing_temp_before = climate_df['Temp_C'].isna().sum() - climate_df['Temp_C'] = climate_df['Temp_C'].ffill().bfill() - missing_temp_after = climate_df['Temp_C'].isna().sum() - if missing_temp_before > 0: - logger.info(f"Filled {missing_temp_before - missing_temp_after} " - f"missing temperature values via forward/backward fill") - - logger.info(f"Climate data fetched successfully: {len(climate_df)} rows") - return climate_df + logger.warning("Meteostat returned no usable data for this location/period") + return None except Exception as e: logger.error(f"Error fetching/processing climate data: {e}") @@ -133,24 +178,14 @@ def fetch_climate_data( def fetch_nearest_station_info( latitude: float, longitude: float, - max_distance_km: float = 100.0 + max_distance_km: float = 150.0, + prefer_recent: bool = True, ) -> Optional[Dict]: """ - Get information about the nearest weather station. + Get information about a nearby weather station. - Useful for understanding data quality and station distance. - - Args: - latitude: Latitude in decimal degrees - longitude: Longitude in decimal degrees - max_distance_km: Maximum search radius in kilometers - - Returns: - Dictionary with station information or None - - Example: - >>> station = fetch_nearest_station_info(47.6593, -117.4491) - >>> print(f"Nearest station: {station['name']} ({station['distance_km']:.1f} km)") + When prefer_recent=True, skip stations whose daily inventory ended before + ~2 years ago so the UI does not claim a dead station is the climate source. """ try: Point, _, Stations = _get_meteostat() @@ -158,40 +193,137 @@ def fetch_nearest_station_info( logger.warning("meteostat not installed - skipping station lookup") return None - location = Point(latitude, longitude) - stations = Stations() - stations = stations.nearby(latitude, longitude) - stations = stations.fetch(1) # Get closest station - - if stations.empty: - logger.warning(f"No weather stations found within {max_distance_km} km") + stations = Stations().nearby(latitude, longitude).fetch(15) + if stations is None or stations.empty: + logger.warning(f"No weather stations found near ({latitude}, {longitude})") return None - station = stations.iloc[0] + if "distance" in stations.columns: + stations = stations.sort_values("distance") + + cutoff = pd.Timestamp.utcnow().tz_localize(None) - pd.Timedelta(days=365 * 2) + chosen = None + for sid, row in stations.iterrows(): + dist_m = row.get("distance", None) + dist_km = (float(dist_m) / 1000.0) if dist_m is not None and pd.notna(dist_m) else None + if dist_km is not None and dist_km > max_distance_km: + continue + if prefer_recent: + raw_end = row.get("daily_end", None) + try: + if raw_end is not None and not pd.isna(raw_end): + if _normalize_ts(raw_end) < cutoff: + continue + except Exception: + pass + chosen = (sid, row, dist_km) + break + + # Fallback: absolute nearest even if historical-only + if chosen is None: + sid = stations.index[0] + row = stations.iloc[0] + dist_m = row.get("distance", 0) + dist_km = float(dist_m) / 1000.0 if dist_m is not None else None + chosen = (sid, row, dist_km) + + sid, station, dist_km = chosen station_dict = { - 'id': station.name, # Station ID - 'name': station.get('name', 'Unknown'), - 'latitude': station.get('latitude', None), - 'longitude': station.get('longitude', None), - 'elevation_m': station.get('elevation', None), - 'distance_km': station.get('distance', 0) / 1000, # Meteostat returns meters + "id": sid, + "name": station.get("name", "Unknown"), + "latitude": station.get("latitude", None), + "longitude": station.get("longitude", None), + "elevation_m": station.get("elevation", None), + "distance_km": dist_km if dist_km is not None else 0.0, } + daily_start = station.get("daily_start", None) + daily_end = station.get("daily_end", None) + if daily_start is not None and not pd.isna(daily_start): + station_dict["daily_start"] = str(daily_start)[:10] + if daily_end is not None and not pd.isna(daily_end): + station_dict["daily_end"] = str(daily_end)[:10] + + logger.info( + f"Nearest station: {station_dict['name']} " + f"({station_dict.get('distance_km', '?')} km away)" + ) + return station_dict - # Include data coverage dates if available - daily_start = station.get('daily_start', None) - daily_end = station.get('daily_end', None) - if daily_start is not None: - station_dict['daily_start'] = str(daily_start)[:10] if daily_start else None - if daily_end is not None: - station_dict['daily_end'] = str(daily_end)[:10] if daily_end else None + except Exception as e: + logger.error(f"Error fetching station info: {e}") + return None - logger.info(f"Nearest station: {station_dict['name']} " - f"({station_dict.get('distance_km', '?')} km away)") - return station_dict +def fetch_open_meteo_climate( + latitude: float, + longitude: float, + start_date: str | pd.Timestamp, + end_date: str | pd.Timestamp, +) -> Optional[pd.DataFrame]: + """ + Fetch historical daily climate from Open-Meteo Archive API (no API key). + + Robust primary source when Meteostat's nearest station has no modern data. + Returns Temp_C / Precip_mm with UTC DatetimeIndex. + """ + try: + import requests + + if isinstance(start_date, pd.Timestamp): + start_date = start_date.strftime("%Y-%m-%d") + if isinstance(end_date, pd.Timestamp): + end_date = end_date.strftime("%Y-%m-%d") + + logger.info( + f"Fetching Open-Meteo historical: {latitude:.4f}, {longitude:.4f} " + f"{start_date} to {end_date}" + ) + + url = "https://archive-api.open-meteo.com/v1/archive" + params = { + "latitude": latitude, + "longitude": longitude, + "start_date": start_date, + "end_date": end_date, + "daily": "temperature_2m_mean,precipitation_sum", + "timezone": "UTC", + } + resp = requests.get(url, params=params, timeout=45) + if resp.status_code != 200: + logger.warning(f"Open-Meteo returned {resp.status_code}: {resp.text[:200]}") + return None + + payload = resp.json() + daily = payload.get("daily") or {} + if "time" not in daily: + logger.warning("Open-Meteo returned no daily time axis") + return None + + df = pd.DataFrame( + { + "date": pd.to_datetime(daily["time"]), + "Temp_C": daily.get("temperature_2m_mean"), + "Precip_mm": daily.get("precipitation_sum"), + } + ).set_index("date") + if df.index.tz is None: + df.index = df.index.tz_localize("UTC") + else: + df.index = df.index.tz_convert("UTC") + + if "Precip_mm" in df.columns: + df["Precip_mm"] = df["Precip_mm"].fillna(0) + if "Temp_C" in df.columns: + df["Temp_C"] = df["Temp_C"].ffill().bfill() + df = df.dropna(how="all") + if df.empty: + return None + + logger.info(f"Open-Meteo historical fetched: {len(df)} days") + return df except Exception as e: - logger.error(f"Error fetching station info: {e}") + logger.error(f"Open-Meteo historical fetch failed: {e}") return None @@ -251,98 +383,3 @@ def merge_discharge_climate( logger.warning("Merged data is empty - check date ranges overlap") return merged - - -# ============================================================================= -# Open-Meteo Historical (strong additional source for historical coverage) -# ============================================================================= - -def fetch_open_meteo_climate( - latitude: float, - longitude: float, - start_date: str | pd.Timestamp, - end_date: str | pd.Timestamp, -) -> Optional[pd.DataFrame]: - """ - Fetch historical daily climate data from Open-Meteo Archive API. - - This is a lightweight, no-key, highly reliable source with excellent - historical depth (often back to 1940s). Excellent complement to Meteostat - for gages where station coverage is poor. - - Returns data normalized to the same schema as other climate sources - (Temp_C, Precip_mm) for easy integration. - - Args: - latitude: Decimal degrees - longitude: Decimal degrees - start_date: YYYY-MM-DD or Timestamp - end_date: YYYY-MM-DD or Timestamp - - Returns: - DataFrame with UTC DatetimeIndex and Temp_C / Precip_mm, or None - """ - try: - import requests - from datetime import datetime - - # Normalize dates - if isinstance(start_date, pd.Timestamp): - start_date = start_date.strftime("%Y-%m-%d") - if isinstance(end_date, pd.Timestamp): - end_date = end_date.strftime("%Y-%m-%d") - - logger.info(f"Fetching Open-Meteo historical: {latitude:.4f}, {longitude:.4f} " - f"{start_date} to {end_date}") - - url = "https://archive-api.open-meteo.com/v1/archive" - params = { - "latitude": latitude, - "longitude": longitude, - "start_date": start_date, - "end_date": end_date, - "daily": "temperature_2m_mean,precipitation_sum", - "timezone": "UTC", - } - - resp = requests.get(url, params=params, timeout=30) - if resp.status_code != 200: - logger.warning(f"Open-Meteo returned {resp.status_code}") - return None - - data = resp.json() - daily = data.get("daily", {}) - - if not daily or "time" not in daily: - logger.warning("Open-Meteo returned no daily data") - return None - - df = pd.DataFrame({ - "date": pd.to_datetime(daily["time"]), - "Temp_C": daily.get("temperature_2m_mean"), - "Precip_mm": daily.get("precipitation_sum"), - }) - - df = df.set_index("date") - df.index = df.index.tz_localize("UTC") - - # Fill missing precip with 0 (common convention) - if "Precip_mm" in df.columns: - df["Precip_mm"] = df["Precip_mm"].fillna(0) - - # Light fill for temperature - if "Temp_C" in df.columns: - df["Temp_C"] = df["Temp_C"].ffill().bfill() - - # Drop completely empty rows - df = df.dropna(how="all") - - if df.empty: - return None - - logger.info(f"Open-Meteo historical fetched: {len(df)} days") - return df - - except Exception as e: - logger.error(f"Open-Meteo historical fetch failed: {e}") - return None diff --git a/hydrology/data/groundwater.py b/hydrology/data/groundwater.py new file mode 100644 index 0000000..c542593 --- /dev/null +++ b/hydrology/data/groundwater.py @@ -0,0 +1,218 @@ +"""Public groundwater data helpers. + +This module starts with USGS field measurements only. Ecology EIM and well-log +sources need separate source-specific modules because their privacy and +analysis-eligibility rules differ. +""" + +from __future__ import annotations + +from typing import Iterable + +import pandas as pd + +from ..core.logging_setup import get_logger + +logger = get_logger(__name__) + +USGS_SOURCE = "USGS" +USGS_FIELD_MEASUREMENTS = "usgs_field_measurements" +DEPTH_TO_WATER_PARAM_CODES = {"72019"} +WATER_LEVEL_ELEVATION_PARAM_CODES = {"62610", "62611", "62612"} +DEFAULT_GROUNDWATER_PARAMETER_CODES = sorted( + DEPTH_TO_WATER_PARAM_CODES | WATER_LEVEL_ELEVATION_PARAM_CODES +) + +SAFE_GROUNDWATER_COLUMNS = [ + "site_id", + "monitoring_location_id", + "source", + "source_type", + "measurement_date", + "parameter_code", + "parameter_name", + "depth_to_water_ft", + "water_level_ft", + "unit", + "vertical_datum", + "approval_status", + "qualifier", + "measuring_agency", + "latitude", + "longitude", + "location_precision", + "data_use", + "analysis_eligible", + "notes", +] + + +def _as_list(value: str | Iterable[str] | None) -> list[str] | None: + if value is None: + return None + if isinstance(value, str): + return [value] + return [str(item) for item in value] + + +def _usgs_monitoring_location_id(site_id: str) -> str: + site = str(site_id) + return site if site.upper().startswith("USGS-") else f"USGS-{site}" + + +def _strip_usgs_prefix(monitoring_location_id: object) -> str | None: + if pd.isna(monitoring_location_id): + return None + text = str(monitoring_location_id) + return text.split("-", 1)[1] if text.upper().startswith("USGS-") else text + + +def _time_interval(start_date: str | None, end_date: str | None) -> str | None: + if start_date and end_date: + return f"{start_date}/{end_date}" + if start_date: + return f"{start_date}/.." + if end_date: + return f"../{end_date}" + return None + + +def _first_existing(row: pd.Series, names: list[str]): + for name in names: + if name in row and pd.notna(row[name]): + return row[name] + return None + + +def _geometry_lon_lat(geometry): + if geometry is None: + return None, None + try: + if pd.isna(geometry): + return None, None + except (TypeError, ValueError): + pass + try: + if getattr(geometry, "geom_type", None) == "Point": + return float(geometry.x), float(geometry.y) + except Exception: + return None, None + return None, None + + +def _location_from_row(row: pd.Series) -> tuple[float | None, float | None]: + lon = _first_existing(row, ["longitude", "lon", "dec_long_va", "x"]) + lat = _first_existing(row, ["latitude", "lat", "dec_lat_va", "y"]) + if lon is None or lat is None: + geom_lon, geom_lat = _geometry_lon_lat(row.get("geometry")) + lon = lon if lon is not None else geom_lon + lat = lat if lat is not None else geom_lat + try: + return float(lat), float(lon) + except (TypeError, ValueError): + return None, None + + +def _safe_numeric(value): + return pd.to_numeric(pd.Series([value]), errors="coerce").iloc[0] + + +def _normalize_row(row: pd.Series) -> dict: + monitoring_location_id = _first_existing(row, ["monitoring_location_id", "monitoringLocationId", "site_no"]) + parameter_code = _first_existing(row, ["parameter_code", "parameterCode", "parm_cd"]) + parameter_code = str(parameter_code).split(".", 1)[0] if parameter_code is not None else None + measurement_date = pd.to_datetime( + _first_existing(row, ["time", "measurement_date", "dateTime", "datetime"]), + errors="coerce", + ) + value = _safe_numeric(_first_existing(row, ["value", "lev_va", "result_va"])) + lat, lon = _location_from_row(row) + + depth_to_water_ft = value if parameter_code in DEPTH_TO_WATER_PARAM_CODES else pd.NA + water_level_ft = value if parameter_code in WATER_LEVEL_ELEVATION_PARAM_CODES else pd.NA + has_value = pd.notna(depth_to_water_ft) or pd.notna(water_level_ft) + has_date = pd.notna(measurement_date) + has_site = monitoring_location_id is not None + + note_parts = [] + if not has_site: + note_parts.append("missing site ID") + if not has_date: + note_parts.append("missing measurement date") + if not has_value: + note_parts.append("missing supported groundwater measurement") + + return { + "site_id": _strip_usgs_prefix(monitoring_location_id), + "monitoring_location_id": monitoring_location_id, + "source": USGS_SOURCE, + "source_type": USGS_FIELD_MEASUREMENTS, + "measurement_date": measurement_date if has_date else pd.NaT, + "parameter_code": parameter_code, + "parameter_name": _first_existing(row, ["parameter_name", "parameterName", "parameter_description"]), + "depth_to_water_ft": depth_to_water_ft, + "water_level_ft": water_level_ft, + "unit": _first_existing(row, ["unit_of_measure", "unitOfMeasure", "unit"]), + "vertical_datum": _first_existing(row, ["vertical_datum", "verticalDatum"]), + "approval_status": _first_existing(row, ["approval_status", "approvalStatus"]), + "qualifier": _first_existing(row, ["qualifier"]), + "measuring_agency": _first_existing(row, ["measuring_agency", "measuringAgency"]), + "latitude": lat, + "longitude": lon, + "location_precision": "measured" if lat is not None and lon is not None else "unavailable", + "data_use": "analysis" if has_value and has_date and has_site else "limited", + "analysis_eligible": bool(has_value and has_date and has_site), + "notes": "; ".join(note_parts), + } + + +def normalize_usgs_groundwater_measurements(raw: pd.DataFrame | None) -> pd.DataFrame: + """Normalize USGS field measurements into HydroPlot's safe groundwater schema.""" + if raw is None or raw.empty: + return pd.DataFrame(columns=SAFE_GROUNDWATER_COLUMNS) + + rows = [_normalize_row(row) for _, row in raw.iterrows()] + normalized = pd.DataFrame(rows, columns=SAFE_GROUNDWATER_COLUMNS) + if not normalized.empty: + normalized["measurement_date"] = pd.to_datetime(normalized["measurement_date"], errors="coerce") + normalized = normalized.sort_values(["site_id", "measurement_date"], na_position="last").reset_index(drop=True) + return normalized + + +def fetch_usgs_groundwater_measurements( + site_ids: str | Iterable[str] | None = None, + *, + bbox: list[float] | None = None, + start_date: str | None = None, + end_date: str | None = None, + parameter_codes: Iterable[str] | None = None, + limit: int | None = 50000, + skip_geometry: bool = False, +) -> pd.DataFrame: + """Fetch public USGS groundwater field measurements and return safe normalized rows.""" + if site_ids is None and bbox is None: + raise ValueError("Provide site_ids or bbox for groundwater field-measurement retrieval") + + try: + from dataretrieval import waterdata + except ImportError as exc: + raise ImportError( + "Install `dataretrieval` to fetch USGS groundwater field measurements." + ) from exc + + monitoring_location_id = None + if site_ids is not None: + monitoring_location_id = [_usgs_monitoring_location_id(site_id) for site_id in _as_list(site_ids)] + + params = { + "monitoring_location_id": monitoring_location_id, + "parameter_code": list(parameter_codes or DEFAULT_GROUNDWATER_PARAMETER_CODES), + "time": _time_interval(start_date, end_date), + "bbox": bbox, + "limit": limit, + "skip_geometry": skip_geometry, + } + params = {key: value for key, value in params.items() if value is not None} + + raw, _metadata = waterdata.get_field_measurements(**params) + return normalize_usgs_groundwater_measurements(raw) diff --git a/hydrology/data/hyriver.py b/hydrology/data/hyriver.py index b6ec92e..77d624d 100644 --- a/hydrology/data/hyriver.py +++ b/hydrology/data/hyriver.py @@ -33,6 +33,7 @@ _boundary_cache: Dict[str, Any] = {} _characteristics_cache: Dict[str, Dict] = {} _flowlines_cache: Dict[Tuple[str, float], Any] = {} +_navigation_flowlines_cache: Dict[Tuple[str, str, float], Any] = {} _dams_cache: Dict[Tuple[str, float], Any] = {} @@ -125,6 +126,54 @@ def get_flowlines(site_id: str, distance_km: float = 50) -> Optional[Any]: return None +def get_navigation_flowlines( + site_id: str, + navigation: str = "upstreamMain", + distance_km: float = 50, +) -> Optional[Any]: + """ + Get NHDPlus flowlines for a specific NLDI navigation mode. + + Args: + site_id: USGS site identifier + navigation: NLDI navigation mode, such as upstreamMain or upstreamTributaries + distance_km: Navigation distance to retrieve + + Returns: + geopandas GeoDataFrame with flowline geometries, or None + """ + cache_key = (site_id, navigation, float(distance_km)) + if cache_key in _navigation_flowlines_cache: + return _navigation_flowlines_cache[cache_key] + + try: + from pynhd import NLDI + + nldi = NLDI() + flowlines = nldi.navigate_byid( + fsource="nwissite", + fid=f"USGS-{site_id}", + navigation=navigation, + source="flowlines", + distance=distance_km, + ) + + if flowlines is not None and not flowlines.empty: + _navigation_flowlines_cache[cache_key] = flowlines + logger.info(f"Retrieved {len(flowlines)} {navigation} flowlines for {site_id}") + return flowlines + + _navigation_flowlines_cache[cache_key] = None + return None + + except ImportError: + logger.error("pynhd not installed") + return None + except Exception as e: + logger.error(f"Error fetching {navigation} flowlines for {site_id}: {e}") + return None + + def get_basin_characteristics(site_id: str) -> Optional[Dict[str, Any]]: """ Get basin characteristics including drainage area, land cover, and soil. @@ -211,6 +260,27 @@ def get_basin_characteristics(site_id: str) -> Optional[Dict[str, Any]]: return result if result else None +def _daymet_credentials_available() -> bool: + """Daymet/ORNL THREDDS requires NASA Earthdata login (401 without it).""" + import os + from pathlib import Path + + if os.environ.get("EARTHDATA_USERNAME") or os.environ.get("EARTHDATA_USER"): + return True + if os.environ.get("EARTHDATA_PASSWORD") or os.environ.get("EARTHDATA_TOKEN"): + return True + # netrc is the usual Earthdata CLI auth location + for candidate in (Path.home() / ".netrc", Path.home() / "_netrc"): + try: + if candidate.is_file() and "urs.earthdata.nasa.gov" in candidate.read_text( + encoding="utf-8", errors="ignore" + ): + return True + except Exception: + pass + return False + + def get_daymet_climate( site_id: str, start_date: str, @@ -236,6 +306,15 @@ def get_daymet_climate( if variables is None: variables = ['prcp', 'tmin', 'tmax'] + # Avoid slow watershed + 401 storm when Earthdata is not configured. + if not _daymet_credentials_available(): + logger.info( + "Skipping Daymet for %s: NASA Earthdata credentials not configured " + "(set EARTHDATA_USERNAME/PASSWORD or ~/.netrc for urs.earthdata.nasa.gov)", + site_id, + ) + return None + # Get watershed boundary for spatial averaging basin = get_watershed_boundary(site_id) if basin is None: diff --git a/hydrology/data/nwm.py b/hydrology/data/nwm.py index 5fcdc6e..00cc55b 100644 --- a/hydrology/data/nwm.py +++ b/hydrology/data/nwm.py @@ -123,6 +123,8 @@ class NWMUSGSComparison: correlation: float nash_sutcliffe: float percent_bias: float + nwm_data: Optional[pd.DataFrame] = None + usgs_data: Optional[pd.DataFrame] = None def to_dict(self) -> Dict[str, Any]: """Convert to dictionary.""" @@ -700,7 +702,9 @@ def compare_nwm_usgs( rmse=rmse, correlation=correlation, nash_sutcliffe=nse, - percent_bias=pbias + percent_bias=pbias, + nwm_data=nwm_data[['streamflow_cfs']].sort_index(), + usgs_data=usgs_data[['usgs_cfs']].sort_index(), ) diff --git a/hydrology/visualization/interactive.py b/hydrology/visualization/interactive.py index e1ba973..f3f79ae 100644 --- a/hydrology/visualization/interactive.py +++ b/hydrology/visualization/interactive.py @@ -19,6 +19,63 @@ logger = get_logger(__name__) +def _discharge_yaxis_layout(values, title: str = "Discharge (cfs)") -> dict: + """Build a Plotly y-axis config that fits the data instead of a fixed log span. + + Uses log only when the positive data truly spans multiple orders of magnitude. + Otherwise linear with padding so moderate-range FDCs/hydrographs are readable. + """ + arr = np.asarray(values, dtype=float) + arr = arr[np.isfinite(arr)] + if arr.size == 0: + return {"title": title, "type": "linear", "rangemode": "tozero"} + + pos = arr[arr > 0] + if pos.size == 0: + # All non-positive — stay linear and pin near zero + lo, hi = float(np.min(arr)), float(np.max(arr)) + if lo == hi: + pad = abs(lo) * 0.1 + 1.0 + return {"title": title, "type": "linear", "range": [lo - pad, hi + pad]} + pad = (hi - lo) * 0.08 + return {"title": title, "type": "linear", "range": [lo - pad, hi + pad]} + + # Use robust high/low so a few zeros or spikes do not crush the scale + q_lo = float(np.nanpercentile(pos, 1)) + q_hi = float(np.nanpercentile(pos, 99.5)) + q_lo = max(q_lo, float(np.min(pos))) + q_hi = max(q_hi, float(np.max(pos)), q_lo * 1.01) + span_ratio = q_hi / max(q_lo, 1e-9) + + # Multi-order FDC (flashy basins): log, but clamp range to the data envelope + if span_ratio >= 40 and q_lo > 0: + log_lo = np.log10(max(q_lo * 0.85, 1e-3)) + log_hi = np.log10(q_hi * 1.15) + if log_hi <= log_lo: + log_hi = log_lo + 0.5 + return { + "title": title, + "type": "log", + "range": [log_lo, log_hi], + "tickformat": "~s", + "exponentformat": "power", + } + + # Typical moderate range: linear, data-driven + lo = float(np.min(arr[arr >= 0])) if np.any(arr >= 0) else float(np.min(arr)) + hi = float(np.max(arr)) + if hi <= lo: + pad = abs(hi) * 0.1 + 1.0 + return {"title": title, "type": "linear", "range": [max(0.0, lo - pad), hi + pad]} + pad = (hi - lo) * 0.08 + return { + "title": title, + "type": "linear", + "range": [max(0.0, lo - pad), hi + pad], + "rangemode": "tozero" if lo >= 0 and lo < hi * 0.05 else "normal", + } + + def interactive_hydrograph( df_q: pd.DataFrame, df_hist: pd.DataFrame = None, @@ -117,11 +174,11 @@ def interactive_hydrograph( hovertemplate='%{x|%Y-%m-%d}<br>%{y:,.0f} cfs<extra></extra>' )) + yaxis = _discharge_yaxis_layout(q_series.values) fig.update_layout( title=title, xaxis_title="Date", - yaxis_title="Discharge (cfs)", - yaxis_type="log", + yaxis=yaxis, height=450, hovermode='x unified', legend=dict(orientation="h", yanchor="bottom", y=1.02, xanchor="right", x=1), @@ -163,6 +220,7 @@ def interactive_fdc( return fig q = df_q[discharge_col].dropna().sort_index() + y_for_scale = np.asarray(q.values, dtype=float) if color_by_dqdt and len(q) > 2: # Calculate dQ/dt (rate of change) @@ -179,6 +237,7 @@ def interactive_fdc( conditions_sorted = conditions[sorted_idx] n = len(q_sorted) exceedance = np.arange(1, n + 1) / (n + 1) * 100 + y_for_scale = np.asarray(q_sorted, dtype=float) fig = go.Figure() @@ -212,6 +271,7 @@ def interactive_fdc( x=0.5, y=0.5, showarrow=False) return fig + y_for_scale = np.asarray(fdc["discharge"].values, dtype=float) fig = go.Figure() fig.add_trace(go.Scatter( x=fdc['exceedance_pct'], y=fdc['discharge'], @@ -220,19 +280,27 @@ def interactive_fdc( hovertemplate='Exceedance: %{x:.1f}%<br>Flow: %{y:,.0f} cfs<extra></extra>' )) - # Add reference lines for key percentiles + # Add reference lines for key percentiles (use same scale as curve) if not q.empty: - for pct, label in [(10, 'Q10'), (50, 'Q50'), (90, 'Q90')]: - q_val = np.percentile(q.values, 100 - pct) - fig.add_hline(y=q_val, line_dash="dash", line_color="gray", - annotation_text=f"{label}: {q_val:,.0f}", annotation_position="right", - line_width=1, opacity=0.5) + for pct, label in [(10, "Q10"), (50, "Q50"), (90, "Q90")]: + q_val = float(np.percentile(q.values, 100 - pct)) + if q_val <= 0: + continue + fig.add_hline( + y=q_val, + line_dash="dash", + line_color="gray", + annotation_text=f"{label}: {q_val:,.0f}", + annotation_position="right", + line_width=1, + opacity=0.5, + ) + yaxis = _discharge_yaxis_layout(y_for_scale) fig.update_layout( title=title, xaxis_title="Exceedance Probability (%)", - yaxis_title="Discharge (cfs)", - yaxis_type="log", + yaxis=yaxis, xaxis=dict(range=[0, 100]), height=450, margin=dict(l=60, r=80, t=60, b=40), @@ -477,7 +545,7 @@ def interactive_return_period( title: Plot title Returns: - Plotly Figure with log-scale return period axis + Plotly Figure with readable return period axis """ fig = go.Figure() @@ -492,7 +560,7 @@ def interactive_return_period( # Plot fitted distribution curves colors = ['#1f77b4', '#ff7f0e', '#2ca02c', '#d62728', '#9467bd'] - rp_range = np.logspace(np.log10(1.01), np.log10(500), 200) + rp_range = np.linspace(1.01, 500, 300) for i, (name, fit) in enumerate(fitted.items()): quantiles = [] @@ -542,8 +610,14 @@ def interactive_return_period( title=title, xaxis_title="Return Period (years)", yaxis_title="Peak Discharge (cfs)", - xaxis_type="log", yaxis_type="log", + xaxis=dict( + type="linear", + range=[0, 105], + tickmode="array", + tickvals=[2, 5, 10, 25, 50, 100], + ticktext=["2", "5", "10", "25", "50", "100"], + ), height=500, hovermode='closest', legend=dict(orientation="h", yanchor="bottom", y=1.02, xanchor="right", x=1), @@ -725,4 +799,288 @@ def baseflow_waterfall( fig.update_yaxes(title_text="Discharge (cfs)", row=1, col=1) fig.update_yaxes(title_text="Baseflow Index", range=[0, 1], row=2, col=1) + return apply_hydro_theme(fig) + + +def interactive_rating_curve( + df_q: pd.DataFrame, + A: float, + B: float, + H0: float = 0.0, + *, + discharge_col: str = "Discharge_cfs", + stage_col: str = "Gage_Height_ft", + title: str = "Stage–discharge rating", + show_auto_fit: bool = True, + auto_fit: Optional[Dict[str, Any]] = None, + show_residuals: bool = True, + log_q: Optional[bool] = None, + color_by: str = "season", + max_points: int = 4000, +) -> go.Figure: + """ + Interactive stage–discharge scatter with a live adjustable rating curve. + + Points are colored by season (or year). User curve Q = A*(H-H0)^B is solid; + optional auto-fit reference is dashed. Optional residual subplot underneath. + """ + from ..analysis.stage_discharge import ( + evaluate_rating_params, + predict_rating_discharge, + season_labels, + fit_best_rating_curve, + ) + + empty = go.Figure() + empty.add_annotation( + text="No stage–discharge pairs available", + xref="paper", yref="paper", x=0.5, y=0.5, + showarrow=False, font=dict(size=15, color="#9bb4c8"), + ) + apply_hydro_theme(empty) + empty.update_layout(height=420, title=title) + + if df_q is None or df_q.empty: + return empty + if discharge_col not in df_q.columns or stage_col not in df_q.columns: + empty.layout.annotations[0].text = "Need Gage_Height_ft + Discharge_cfs" + return empty + + pairs = df_q[[stage_col, discharge_col]].dropna().copy() + pairs = pairs[(pairs[stage_col] > 0) & (pairs[discharge_col] > 0)] + if len(pairs) < 5: + empty.layout.annotations[0].text = "Need ≥5 stage–discharge pairs" + return empty + + # Downsample dense records for browser snappiness (keep extremes) + if len(pairs) > max_points: + step = max(1, len(pairs) // max_points) + keep = pairs.iloc[::step] + # Always include min/max stage and peak Q + extrema_idx = [ + pairs[stage_col].idxmin(), + pairs[stage_col].idxmax(), + pairs[discharge_col].idxmax(), + ] + pairs = pd.concat([keep, pairs.loc[extrema_idx]]).sort_index() + pairs = pairs[~pairs.index.duplicated(keep="first")] + + H = pairs[stage_col].astype(float) + Q = pairs[discharge_col].astype(float) + scored = evaluate_rating_params(H, Q, A, B, H0) + + if auto_fit is None and show_auto_fit: + try: + auto_fit = fit_best_rating_curve(H, Q, min_points=10) + except Exception: + auto_fit = None + + rows = 2 if show_residuals else 1 + row_heights = [0.72, 0.28] if show_residuals else [1.0] + fig = make_subplots( + rows=rows, + cols=1, + shared_xaxes=True, + row_heights=row_heights, + vertical_spacing=0.06 if show_residuals else 0.02, + subplot_titles=(None, "Residuals (Q_fit − Q_obs)") if show_residuals else None, + ) + + season_colors = { + "DJF": "#5b9bd5", + "MAM": "#70ad47", + "JJA": "#ed7d31", + "SON": "#9e7bb5", + "UNK": "#8a9bb0", + } + + if color_by == "year": + years = pd.DatetimeIndex(pairs.index).year + uniq = sorted(set(int(y) for y in years)) + # Teal→amber ramp + n = max(len(uniq), 1) + for i, yr in enumerate(uniq): + mask = years == yr + t = i / max(n - 1, 1) + r = int(78 + t * (237 - 78)) + g = int(205 + t * (125 - 205)) + b = int(196 + t * (49 - 196)) + color = f"rgb({r},{g},{b})" + fig.add_trace( + go.Scattergl( + x=H.values[mask], + y=Q.values[mask], + mode="markers", + name=str(yr), + marker=dict(size=6, color=color, opacity=0.55, line=dict(width=0)), + hovertemplate=( + f"{yr}<br>H=%{{x:.2f}} ft<br>Q=%{{y:,.1f}} cfs<extra></extra>" + ), + legendgroup="pts", + ), + row=1, col=1, + ) + else: + seasons = season_labels(pairs.index) + for season, color in season_colors.items(): + mask = seasons.values == season + if not np.any(mask): + continue + fig.add_trace( + go.Scattergl( + x=H.values[mask], + y=Q.values[mask], + mode="markers", + name=season, + marker=dict(size=7, color=color, opacity=0.55, line=dict(width=0)), + hovertemplate=( + f"{season}<br>H=%{{x:.2f}} ft<br>Q=%{{y:,.1f}} cfs<extra></extra>" + ), + legendgroup="pts", + ), + row=1, col=1, + ) + + # Curve domain + H0f = float(H0) + H_min = float(H.min()) + H_max = float(H.max()) + H_lo = max(H_min, H0f + 1e-3) if abs(H0f) > 1e-9 else H_min + H_line = np.linspace(H_lo, H_max, 250) + Q_user = predict_rating_discharge(H_line, A, B, H0f) + + r2_txt = f"{scored['R2']:.3f}" if np.isfinite(scored.get("R2", np.nan)) else "—" + fig.add_trace( + go.Scatter( + x=H_line, + y=Q_user, + mode="lines", + name=f"Your fit · R²={r2_txt}", + line=dict(color="#ff6b6b", width=3.2), + hovertemplate="H=%{x:.2f} ft<br>Q_fit=%{y:,.1f} cfs<extra>Your rating</extra>", + ), + row=1, col=1, + ) + + if ( + show_auto_fit + and auto_fit + and auto_fit.get("model") != "none" + and np.isfinite(auto_fit.get("A", np.nan)) + ): + aA, aB, aH0 = auto_fit["A"], auto_fit["B"], float(auto_fit.get("H0") or 0.0) + a_lo = max(H_min, aH0 + 1e-3) if abs(aH0) > 1e-9 else H_min + H_auto = np.linspace(a_lo, H_max, 250) + Q_auto = predict_rating_discharge(H_auto, aA, aB, aH0) + a_r2 = auto_fit.get("R2", np.nan) + a_r2_txt = f"{a_r2:.3f}" if np.isfinite(a_r2) else "—" + fig.add_trace( + go.Scatter( + x=H_auto, + y=Q_auto, + mode="lines", + name=f"Auto-fit · R²={a_r2_txt}", + line=dict(color="#4ecdc4", width=2.2, dash="dash"), + hovertemplate="H=%{x:.2f} ft<br>Q_auto=%{y:,.1f} cfs<extra>Auto-fit</extra>", + ), + row=1, col=1, + ) + + if show_residuals and scored["n_points"] > 0: + resid = scored["residuals"] + fig.add_trace( + go.Scattergl( + x=scored["H_clean"], + y=resid, + mode="markers", + name="Residual", + showlegend=False, + marker=dict( + size=5, + color=resid, + colorscale=[ + [0.0, "#4ecdc4"], + [0.5, "#e7eef7"], + [1.0, "#ff6b6b"], + ], + cmid=0, + opacity=0.65, + line=dict(width=0), + ), + hovertemplate="H=%{x:.2f} ft<br>ΔQ=%{y:,.1f} cfs<extra></extra>", + ), + row=2, col=1, + ) + fig.add_hline( + y=0, line=dict(color="rgba(118,169,192,0.45)", width=1, dash="dot"), + row=2, col=1, + ) + + # Axis type for Q + use_log = log_q + if use_log is None: + q_pos = Q[Q > 0] + use_log = bool(len(q_pos) and (q_pos.max() / max(q_pos.min(), 1e-9)) >= 40) + + yaxis_cfg = _discharge_yaxis_layout(Q.values, title="Discharge (cfs)") + if use_log: + yaxis_cfg["type"] = "log" + else: + yaxis_cfg["type"] = "linear" + + eq = scored.get("equation") or f"Q = {A:.4g}·(H−{H0:.3f})^{B:.3f}" + subtitle = ( + f"<span style='color:#9bb4c8;font-size:12px'>{eq} · n={scored['n_points']:,}</span>" + ) + + fig.update_layout( + title=dict( + text=f"{title}<br>{subtitle}", + font=dict(size=15), + ), + height=560 if show_residuals else 440, + showlegend=True, + legend=dict( + orientation="h", + yanchor="bottom", + y=1.02, + xanchor="right", + x=1, + bgcolor="rgba(16,28,46,0.55)", + bordercolor="rgba(78,205,196,0.25)", + borderwidth=1, + font=dict(size=11), + ), + margin=dict(l=60, r=24, t=90, b=48), + ) + fig.update_xaxes(title_text="Stage height (ft)" if not show_residuals else None, row=1, col=1) + if show_residuals: + fig.update_xaxes(title_text="Stage height (ft)", row=2, col=1) + fig.update_yaxes(title_text="ΔQ (cfs)", row=2, col=1) + fig.update_yaxes(row=1, col=1, **yaxis_cfg) + + return apply_hydro_theme(fig) + + +def apply_hydro_theme(fig: "go.Figure") -> "go.Figure": + """Lightweight consistent theming for Plotly charts to match the dashboard premium visual language. + + Safe to call on any interactive fig returned from this module. + Uses the same teal accent and dark surfaces as the Streamlit custom CSS. + (Optional polish; callers can opt-in without behavior change.) + """ + try: + fig.update_layout( + font=dict(family="Inter, system-ui, -apple-system, Segoe UI, Roboto, sans-serif", size=12, color="#e7eef7"), + paper_bgcolor="rgba(16, 28, 46, 0.85)", + plot_bgcolor="rgba(10, 21, 36, 0.6)", + margin=dict(l=50, r=20, t=60, b=40), + hoverlabel=dict(bgcolor="#101c2e", bordercolor="#4ecdc4", font=dict(color="#e7eef7")), + ) + # Subtle grid using the muted border tone + fig.update_xaxes(gridcolor="rgba(118, 169, 192, 0.12)", zerolinecolor="rgba(118, 169, 192, 0.2)") + fig.update_yaxes(gridcolor="rgba(118, 169, 192, 0.12)", zerolinecolor="rgba(118, 169, 192, 0.2)") + except Exception: + # Never break a chart for theme + pass return fig diff --git a/hydrology/visualization/plots.py b/hydrology/visualization/plots.py index ad5a501..44d8805 100644 --- a/hydrology/visualization/plots.py +++ b/hydrology/visualization/plots.py @@ -1340,17 +1340,17 @@ def plot_double_mass_curve(ax, df_merged: pd.DataFrame = None, config: Dict[str, def plot_rating_curve(ax, df_q: pd.DataFrame = None, config: Dict[str, Any] = None, **kwargs): """ - Plot stage-discharge rating curve with power-law fit: Q = A * H^B. + Plot stage-discharge rating curve with best power-law / offset-power fit. - Uses the fit_powerlaw_rating_curve function to fit the relationship - between stage height and discharge, plotting observed points and fitted curve. + Points are colored by meteorological season (DJF/MAM/JJA/SON). The fit + prefers Q = A*(H-H0)^B when a simple Q = A*H^B is a poor model (common when + gage zero is not the zero-flow stage — e.g. USGS 14018500). - Args: - ax: Matplotlib axis to plot on - df_q: DataFrame with Discharge_cfs and Gage_Height_ft columns - config: Optional configuration dict + Daily mean Q (00060) is paired with daily mean stage (00065) by date. + Note: for continuous gages, published Q is often already rating-derived from + stage, so this is an empirical consistency / hysteresis diagnostic. """ - from ..analysis.stage_discharge import fit_powerlaw_rating_curve + from ..analysis.stage_discharge import fit_best_rating_curve, season_labels cfg = {**DEFAULT_CONFIG, **(config or {})} DISCHARGE_COL = cfg['discharge_col'] @@ -1369,44 +1369,96 @@ def plot_rating_curve(ax, df_q: pd.DataFrame = None, config: Dict[str, Any] = No return try: - df = df_q[[STAGE_COL, DISCHARGE_COL]].dropna() + df = df_q[[STAGE_COL, DISCHARGE_COL]].dropna().copy() df = df[(df[STAGE_COL] > 0) & (df[DISCHARGE_COL] > 0)] if len(df) < 10: _plot_placeholder(ax, "Rating Curve\nInsufficient data (need 10+ points)") return - # Fit power-law rating curve - A, B, R2, Q_pred = fit_powerlaw_rating_curve( - df[STAGE_COL], df[DISCHARGE_COL], min_points=10 - ) - - if np.isnan(A) or np.isnan(B): + fit = fit_best_rating_curve(df[STAGE_COL], df[DISCHARGE_COL], min_points=10) + if fit["model"] == "none" or not np.isfinite(fit.get("A", np.nan)): _plot_placeholder(ax, "Rating Curve\nCould not fit curve") return - # Plot observed data - ax.scatter(df[STAGE_COL], df[DISCHARGE_COL], c='steelblue', s=10, - alpha=0.4, label='Observed') + # Season colors (cool → warm year cycle) + seasons = season_labels(df.index) + season_colors = { + "DJF": "#4C78A8", # winter blue + "MAM": "#59A14F", # spring green + "JJA": "#F28E2B", # summer orange + "SON": "#B07AA1", # fall purple + "UNK": "#9E9E9E", + } + for season, color in season_colors.items(): + mask = seasons.values == season + if not np.any(mask): + continue + ax.scatter( + df.loc[mask, STAGE_COL], + df.loc[mask, DISCHARGE_COL], + c=color, + s=14, + alpha=0.45, + edgecolors="none", + label=season, + zorder=2, + ) - # Plot fitted curve - H_range = np.linspace(df[STAGE_COL].min(), df[STAGE_COL].max(), 100) - Q_fit = A * H_range ** B - ax.plot(H_range, Q_fit, 'r-', linewidth=2.5, - label=f'Q = {A:.3f} * H^{B:.2f} (R²={R2:.3f})') + # Fitted curve + H0 = float(fit.get("H0") or 0.0) + H_min = float(df[STAGE_COL].min()) + H_max = float(df[STAGE_COL].max()) + H_lo = max(H_min, H0 + 1e-3) if fit["model"] == "offset_powerlaw" else H_min + H_range = np.linspace(H_lo, H_max, 200) + if fit["model"] == "offset_powerlaw": + Q_fit = fit["A"] * np.maximum(H_range - H0, 1e-10) ** fit["B"] + else: + Q_fit = fit["A"] * H_range ** fit["B"] + + ax.plot( + H_range, + Q_fit, + color="#E15759", + linewidth=2.4, + zorder=3, + label=f'{fit["equation"]} (R²={fit["R2"]:.3f}, n={fit["n_points"]:,})', + ) - # Formatting - ax.set_xlabel('Stage Height (ft)') - ax.set_ylabel('Discharge (cfs)') - ax.set_yscale('log') - ax.set_xscale('log') - ax.set_title('Stage-Discharge Rating Curve', fontweight='bold') - ax.legend(loc='upper left', fontsize=8) - ax.grid(True, alpha=0.3, which='both') - ax.yaxis.set_major_formatter(mticker.ScalarFormatter()) - ax.yaxis.get_major_formatter().set_scientific(False) - ax.xaxis.set_major_formatter(mticker.ScalarFormatter()) - ax.xaxis.get_major_formatter().set_scientific(False) + # Axes: stage linear (physically natural); Q log only if multi-order span + ax.set_xlabel("Stage height (ft)") + ax.set_ylabel("Discharge (cfs)") + q_pos = df[DISCHARGE_COL][df[DISCHARGE_COL] > 0] + if len(q_pos) and (q_pos.max() / max(q_pos.min(), 1e-9)) >= 40: + ax.set_yscale("log") + ax.yaxis.set_major_formatter(mticker.ScalarFormatter()) + ax.yaxis.get_major_formatter().set_scientific(False) + else: + ax.set_yscale("linear") + y0 = 0.0 + y1 = float(df[DISCHARGE_COL].max()) * 1.08 + ax.set_ylim(y0, y1) + + ax.set_xscale("linear") + ax.set_title("Stage–discharge rating curve (seasonal points)", fontweight="bold") + ax.legend(loc="upper left", fontsize=7, framealpha=0.92) + ax.grid(True, alpha=0.3, which="both") + # Caption-like note + model_note = ( + "Offset power-law (H₀ stage correction)" + if fit["model"] == "offset_powerlaw" + else "Simple power-law" + ) + ax.text( + 0.98, + 0.02, + f"{model_note} · daily mean Q (00060) vs stage (00065)", + transform=ax.transAxes, + ha="right", + va="bottom", + fontsize=7, + color="#666666", + ) except Exception as e: logger.error(f"Rating curve error: {e}") diff --git a/requirements.txt b/requirements.txt index c5c4abb..67a6dd0 100644 --- a/requirements.txt +++ b/requirements.txt @@ -6,6 +6,7 @@ requests>=2.28.0 # Data fetching meteostat==1.6.8 +dataretrieval>=1.0.0 # Visualization matplotlib>=3.7.0 diff --git a/scripts/__init__.py b/scripts/__init__.py new file mode 100644 index 0000000..a14079c --- /dev/null +++ b/scripts/__init__.py @@ -0,0 +1 @@ +"""Command-line helpers for HydroPlot.""" diff --git a/scripts/__main__.py b/scripts/__main__.py new file mode 100644 index 0000000..dae2c46 --- /dev/null +++ b/scripts/__main__.py @@ -0,0 +1,5 @@ +from .run_case_study import main + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/force_restart_streamlit.py b/scripts/force_restart_streamlit.py new file mode 100644 index 0000000..82de79c --- /dev/null +++ b/scripts/force_restart_streamlit.py @@ -0,0 +1,76 @@ +#!/usr/bin/env python3 +import os +import re +import signal +import subprocess +import time +import urllib.request +from pathlib import Path + +ROOT = Path("/home/cam/source/repos/Hydrology") +LOG = ROOT / "outputs" / "logs" / "streamlit.log" +PORT = 8501 + + +def kill_port(port: int) -> None: + try: + out = subprocess.check_output( + ["ss", "-lptn", f"sport = :{port}"], text=True, stderr=subprocess.DEVNULL + ) + except Exception: + out = "" + for pid in re.findall(r"pid=(\d+)", out): + try: + os.kill(int(pid), signal.SIGKILL) + print("killed port holder", pid) + except Exception as e: + print("port kill fail", pid, e) + + +def kill_streamlit() -> None: + out = subprocess.check_output(["ps", "-eo", "pid,cmd"], text=True) + for line in out.splitlines(): + if "streamlit" not in line or "hydrology" not in line: + continue + if "force_restart" in line or "restart_streamlit" in line: + continue + try: + pid = int(line.split(None, 1)[0]) + os.kill(pid, signal.SIGKILL) + print("killed streamlit", pid) + except Exception as e: + print("streamlit kill fail", e) + + +kill_port(PORT) +kill_streamlit() +time.sleep(2) + +LOG.parent.mkdir(parents=True, exist_ok=True) +cmd = [ + str(ROOT / "venv" / "bin" / "streamlit"), + "run", + "hydrology/app/app.py", + "--server.headless", + "true", + "--server.address", + "0.0.0.0", + "--server.port", + str(PORT), +] +with open(LOG, "a", encoding="utf-8") as fh: + fh.write("\n--- force restart ---\n") + subprocess.Popen( + cmd, + cwd=str(ROOT), + stdout=fh, + stderr=subprocess.STDOUT, + start_new_session=True, + ) + +time.sleep(5) +try: + print("http", urllib.request.urlopen(f"http://127.0.0.1:{PORT}/", timeout=8).status) +except Exception as e: + print("http_err", e) + print(LOG.read_text(encoding="utf-8", errors="replace")[-600:]) diff --git a/scripts/restart_streamlit_pi.py b/scripts/restart_streamlit_pi.py new file mode 100644 index 0000000..0c57fed --- /dev/null +++ b/scripts/restart_streamlit_pi.py @@ -0,0 +1,91 @@ +#!/usr/bin/env python3 +"""Restart HydroPlot Streamlit on the Pi without shell self-match on pgrep -f.""" +import os +import signal +import subprocess +import sys +import time +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +PORT = 8501 +LOG = Path("/tmp/streamlit-hydro.log") + + +def _pids_listening_on(port: int) -> list[int]: + try: + out = subprocess.check_output( + ["ss", "-lptn", f"sport = :{port}"], + text=True, + stderr=subprocess.DEVNULL, + ) + except Exception: + out = "" + pids: list[int] = [] + for token in out.replace(",", " ").split(): + if token.startswith("pid="): + try: + pids.append(int(token.split("=")[1])) + except ValueError: + pass + return sorted(set(pids)) + + +def _kill(pids: list[int]) -> None: + for pid in pids: + try: + os.kill(pid, signal.SIGTERM) + except ProcessLookupError: + pass + time.sleep(1.5) + for pid in pids: + try: + os.kill(pid, signal.SIGKILL) + except ProcessLookupError: + pass + + +def main() -> int: + old = _pids_listening_on(PORT) + if old: + print(f"stopping pids on :{PORT}: {old}") + _kill(old) + time.sleep(1) + + venv_streamlit = ROOT / "venv" / "bin" / "streamlit" + if not venv_streamlit.exists(): + print("streamlit binary missing", file=sys.stderr) + return 1 + + env = os.environ.copy() + env["PYTHONPATH"] = str(ROOT) + log_f = open(LOG, "w", encoding="utf-8") + proc = subprocess.Popen( + [ + str(venv_streamlit), + "run", + "hydrology/app/app.py", + "--server.headless", + "true", + "--server.address", + "0.0.0.0", + "--server.port", + str(PORT), + ], + cwd=str(ROOT), + env=env, + stdout=log_f, + stderr=subprocess.STDOUT, + start_new_session=True, + ) + time.sleep(3) + print(f"started pid={proc.pid}") + try: + print(LOG.read_text(encoding="utf-8", errors="replace")[-800:]) + except Exception as exc: + print(f"(log read failed: {exc})") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/run_case_study.py b/scripts/run_case_study.py new file mode 100644 index 0000000..418a051 --- /dev/null +++ b/scripts/run_case_study.py @@ -0,0 +1,125 @@ +from __future__ import annotations + +import sys +from pathlib import Path + +import pandas as pd +import yaml + +from hydrology.analysis.baseflow import eckhardt_filter, lyne_hollick_filter +from hydrology.analysis.signatures import compute_hydrologic_signatures +from hydrology.analysis.validation import validate_range, validate_relative_error +from hydrology.data.usgs import fetch_discharge_data + + +def _load_daily_flow(config: dict, config_path: Path) -> pd.Series | None: + flow_csv = config.get("input_daily_flow_csv") + if not flow_csv: + site_ids = config.get("site_ids", []) + if not site_ids: + return None + df = fetch_discharge_data(str(site_ids[0]), config.get("start_date"), config.get("end_date")) + if df is None or df.empty: + return None + return pd.Series(df["Discharge_cfs"].astype(float).values, index=df.index, name="discharge_cfs") + + path = Path(flow_csv) + if not path.is_absolute(): + path = config_path.parent / path + df = pd.read_csv(path, parse_dates=["date"]) + return pd.Series(df["discharge_cfs"].astype(float).values, index=df["date"], name="discharge_cfs") + + +def _write_flow_outputs(flow: pd.Series, analyses: list[str], output_dir: Path) -> dict[str, float]: + metrics: dict[str, float] = {"n_days": float(flow.dropna().shape[0])} + + if "baseflow" in analyses: + lh = lyne_hollick_filter(flow) + eckhardt = eckhardt_filter(flow) + metrics["baseflow_index_lh"] = lh.bfi + metrics["baseflow_index_eckhardt"] = eckhardt.bfi + metrics["baseflow_index_difference"] = abs(lh.bfi - eckhardt.bfi) + components = lh.components.rename(columns={"baseflow": "baseflow_lyne_hollick"}) + components["baseflow_eckhardt"] = eckhardt.components["baseflow"] + components["quickflow_lyne_hollick"] = components["total_flow"] - components["baseflow_lyne_hollick"] + components.to_csv(output_dir / "baseflow_components.csv", index_label="date") + + if "signatures" in analyses: + signatures = compute_hydrologic_signatures(flow) + metrics.update({key: float(value) for key, value in signatures.items() if pd.notna(value)}) + pd.DataFrame([signatures]).to_csv(output_dir / "signatures.csv", index=False) + + return metrics + + +def _write_validation_summary(config: dict, metrics: dict[str, float], output_dir: Path) -> None: + rows = [] + for check in config.get("validation", []): + metric = check["metric"] + if metric not in metrics: + rows.append( + { + "metric": metric, + "status": "MISSING", + "value": "", + "expected": "", + "relative_error": "", + "message": f"{metric}: metric was not produced by configured analyses", + } + ) + continue + + value = float(metrics[metric]) + if "lower" in check and "upper" in check: + result = validate_range(metric, value, float(check["lower"]), float(check["upper"])) + else: + result = validate_relative_error(metric, value, float(check["expected"]), float(check.get("tolerance", 0.0))) + rows.append( + { + "metric": result.metric, + "status": result.status, + "value": result.value, + "expected": result.expected, + "relative_error": "" if result.relative_error is None else result.relative_error, + "message": result.message, + } + ) + + if rows: + pd.DataFrame(rows).to_csv(output_dir / "validation_summary.csv", index=False) + + +def run_case_study(config_path: Path) -> Path: + """Run a lightweight case config and write a summary artifact.""" + config = yaml.safe_load(config_path.read_text(encoding="utf-8")) + output_dir = config_path.parent / "outputs" + output_dir.mkdir(exist_ok=True) + summary = output_dir / "run_summary.md" + analyses = config.get("analyses", []) + flow = _load_daily_flow(config, config_path) + metrics: dict[str, float] = {} + if flow is not None: + metrics.update(_write_flow_outputs(flow, analyses, output_dir)) + _write_validation_summary(config, metrics, output_dir) + summary.write_text( + ( + f"# {config['case']} run summary\n\n" + f"Configured analyses: {', '.join(analyses)}\n\n" + f"Validation checks: {len(config.get('validation', []))}\n" + ), + encoding="utf-8", + ) + return summary + + +def main() -> int: + if len(sys.argv) != 2: + print("Usage: python scripts/run_case_study.py docs/cases/<case>/case.yml") + return 2 + summary = run_case_study(Path(sys.argv[1])) + print(f"Wrote {summary}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/spokane_flood_explorer.py b/spokane_flood_explorer.py index 21738ff..1314d4b 100644 --- a/spokane_flood_explorer.py +++ b/spokane_flood_explorer.py @@ -3,7 +3,7 @@ A companion to the ArcGIS Experience Builder app that provides historical streamflow analysis with time controls for Spokane County -USGS gauges. +USGS gages. Run with: streamlit run spokane_flood_explorer.py """ @@ -42,9 +42,9 @@ ) # ============================================================================= -# SPOKANE COUNTY GAUGES +# SPOKANE COUNTY GAGES # ============================================================================= -SPOKANE_GAUGES = { +SPOKANE_GAGES = { "12422500": { "name": "Spokane River at Spokane", "lat": 47.6588, @@ -136,7 +136,7 @@ .status-normal { color: #00e676; } .status-action { color: #ffab40; } .status-flood { color: #ff5252; } - .gauge-header { + .gage-header { background: linear-gradient(90deg, #1a1a2e, #16213e); border-left: 4px solid #00d4ff; padding: 10px 15px; @@ -207,7 +207,7 @@ def load_peaks(site_id: str) -> pd.DataFrame: ) -def create_hydrograph(data: pd.DataFrame, gauge_info: dict, site_id: str) -> go.Figure: +def create_hydrograph(data: pd.DataFrame, gage_info: dict, site_id: str) -> go.Figure: """Create an interactive hydrograph with flood thresholds.""" fig = go.Figure() @@ -217,17 +217,17 @@ def create_hydrograph(data: pd.DataFrame, gauge_info: dict, site_id: str) -> go. y=data["discharge_cfs"], mode="lines", name="Discharge", - line=dict(color=gauge_info["color"], width=1.5), + line=dict(color=gage_info["color"], width=1.5), fill="tozeroy", - fillcolor=f"rgba({int(gauge_info['color'][1:3], 16)}, " - f"{int(gauge_info['color'][3:5], 16)}, " - f"{int(gauge_info['color'][5:7], 16)}, 0.15)", + fillcolor=f"rgba({int(gage_info['color'][1:3], 16)}, " + f"{int(gage_info['color'][3:5], 16)}, " + f"{int(gage_info['color'][5:7], 16)}, 0.15)", hovertemplate="<b>%{x|%b %d, %Y}</b><br>Discharge: %{y:,.0f} cfs<extra></extra>", )) # Flood stage line fig.add_hline( - y=gauge_info["flood_stage_cfs"], + y=gage_info["flood_stage_cfs"], line_dash="dash", line_color="#ff5252", annotation_text="Flood Stage", @@ -237,7 +237,7 @@ def create_hydrograph(data: pd.DataFrame, gauge_info: dict, site_id: str) -> go. # Action stage line fig.add_hline( - y=gauge_info["action_stage_cfs"], + y=gage_info["action_stage_cfs"], line_dash="dot", line_color="#ffab40", annotation_text="Action Stage", @@ -248,8 +248,8 @@ def create_hydrograph(data: pd.DataFrame, gauge_info: dict, site_id: str) -> go. fig.update_layout( **PLOTLY_LAYOUT, title=dict( - text=f"<b>{gauge_info['name']}</b> — USGS {site_id}", - font=dict(size=16, color=gauge_info["color"]), + text=f"<b>{gage_info['name']}</b> — USGS {site_id}", + font=dict(size=16, color=gage_info["color"]), ), yaxis_title="Discharge (cfs)", xaxis_title="", @@ -262,7 +262,7 @@ def create_hydrograph(data: pd.DataFrame, gauge_info: dict, site_id: str) -> go. def create_comparison_chart(all_data: dict) -> go.Figure: - """Create a multi-gauge comparison chart.""" + """Create a multi-gage comparison chart.""" fig = go.Figure() for site_id, (df, info) in all_data.items(): @@ -295,7 +295,7 @@ def create_comparison_chart(all_data: dict) -> go.Figure: fig.update_layout( **PLOTLY_LAYOUT, - title=dict(text="<b>Multi-Gauge Comparison</b> — % of Flood Stage", font=dict(size=16)), + title=dict(text="<b>Multi-Gage Comparison</b> — % of Flood Stage", font=dict(size=16)), yaxis_title="% of Flood Stage", height=450, xaxis_rangeslider_visible=True, @@ -305,7 +305,7 @@ def create_comparison_chart(all_data: dict) -> go.Figure: return fig -def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str) -> go.Figure: +def create_annual_peak_chart(peaks: pd.DataFrame, gage_info: dict, site_id: str) -> go.Figure: """Create an annual peak flood chart.""" if peaks.empty or "peak_discharge_cfs" not in peaks.columns: return None @@ -318,12 +318,12 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str colors = [] for _, row in peaks_sorted.iterrows(): q = row["peak_discharge_cfs"] - if q >= gauge_info["flood_stage_cfs"]: + if q >= gage_info["flood_stage_cfs"]: colors.append("#ff5252") - elif q >= gauge_info["action_stage_cfs"]: + elif q >= gage_info["action_stage_cfs"]: colors.append("#ffab40") else: - colors.append(gauge_info["color"]) + colors.append(gage_info["color"]) fig.add_trace(go.Bar( x=peaks_sorted["peak_date"], @@ -336,7 +336,7 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str )) fig.add_hline( - y=gauge_info["flood_stage_cfs"], + y=gage_info["flood_stage_cfs"], line_dash="dash", line_color="#ff5252", annotation_text="Flood Stage", @@ -346,8 +346,8 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str fig.update_layout( **PLOTLY_LAYOUT, title=dict( - text=f"<b>Annual Peak Floods</b> — {gauge_info['name']}", - font=dict(size=16, color=gauge_info["color"]), + text=f"<b>Annual Peak Floods</b> — {gage_info['name']}", + font=dict(size=16, color=gage_info["color"]), ), yaxis_title="Peak Discharge (cfs)", height=350, @@ -363,16 +363,16 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str st.markdown("## 🌊 Spokane Flood Explorer") st.markdown("---") - # Gauge selection - st.markdown("### Select Gauges") - selected_gauges = {} - for site_id, info in SPOKANE_GAUGES.items(): + # Gage selection + st.markdown("### Select Gages") + selected_gages = {} + for site_id, info in SPOKANE_GAGES.items(): if st.checkbox( f"{info['name']}", value=(site_id in ["12422500", "12424000"]), key=f"chk_{site_id}" ): - selected_gauges[site_id] = info + selected_gages[site_id] = info st.markdown("---") @@ -410,8 +410,8 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str # View mode view_mode = st.radio("View Mode", [ - "Individual Gauges", - "Multi-Gauge Comparison", + "Individual Gages", + "Multi-Gage Comparison", "Flood History", ], index=0) @@ -436,26 +436,26 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str ) st.markdown( '<p style="text-align:center; color:#8892b0; margin-bottom: 30px;">' - 'Historical streamflow analysis for Spokane County USGS gauges • ' + 'Historical streamflow analysis for Spokane County USGS gages • ' f'Showing {start_date.strftime("%b %Y")} — {end_date.strftime("%b %Y")}' '</p>', unsafe_allow_html=True ) -if not selected_gauges: - st.warning("Select at least one gauge from the sidebar to get started.") +if not selected_gages: + st.warning("Select at least one gage from the sidebar to get started.") st.stop() -# Fetch data for all selected gauges -all_gauge_data = {} -for site_id, info in selected_gauges.items(): +# Fetch data for all selected gages +all_gage_data = {} +for site_id, info in selected_gages.items(): df = load_discharge(site_id, start_date.isoformat(), end_date.isoformat()) - all_gauge_data[site_id] = (df, info) + all_gage_data[site_id] = (df, info) # ============================================================================= -# GAUGE MAP +# GAGE MAP # ============================================================================= -st.markdown("### 📍 Gauge Locations") +st.markdown("### 📍 Gage Locations") m = folium.Map( location=[47.66, -117.35], @@ -463,8 +463,8 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str tiles="CartoDB dark_matter", ) -for site_id, info in selected_gauges.items(): - df, _ = all_gauge_data.get(site_id, (None, None)) +for site_id, info in selected_gages.items(): + df, _ = all_gage_data.get(site_id, (None, None)) latest = "N/A" if df is not None and not df.empty: latest = f"{df['discharge_cfs'].iloc[-1]:,.0f} cfs" @@ -491,8 +491,8 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str # ============================================================================= # METRIC CARDS # ============================================================================= -cols = st.columns(len(selected_gauges)) -for i, (site_id, (df, info)) in enumerate(all_gauge_data.items()): +cols = st.columns(len(selected_gages)) +for i, (site_id, (df, info)) in enumerate(all_gage_data.items()): with cols[i]: if df is not None and not df.empty: latest = df["discharge_cfs"].iloc[-1] @@ -521,8 +521,8 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str # ============================================================================= # CHARTS # ============================================================================= -if view_mode == "Individual Gauges": - for site_id, (df, info) in all_gauge_data.items(): +if view_mode == "Individual Gages": + for site_id, (df, info) in all_gage_data.items(): if df is not None and not df.empty: fig = create_hydrograph(df, info, site_id) st.plotly_chart(fig, use_container_width=True) @@ -546,12 +546,12 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str else: st.warning(f"No data available for {info['name']} (USGS {site_id})") -elif view_mode == "Multi-Gauge Comparison": - fig = create_comparison_chart(all_gauge_data) +elif view_mode == "Multi-Gage Comparison": + fig = create_comparison_chart(all_gage_data) st.plotly_chart(fig, use_container_width=True) st.markdown( - "*Each gauge is normalized to its flood stage threshold so you can " + "*Each gage is normalized to its flood stage threshold so you can " "compare relative flood risk across different rivers.*" ) @@ -562,7 +562,7 @@ def create_annual_peak_chart(peaks: pd.DataFrame, gauge_info: dict, site_id: str "the single highest instantaneous discharge each water year." ) - for site_id, info in selected_gauges.items(): + for site_id, info in selected_gages.items(): peaks = load_peaks(site_id) if peaks is not None and not peaks.empty: diff --git a/tests/test_app_indicators.py b/tests/test_app_indicators.py index 5659f29..bff2ea5 100644 --- a/tests/test_app_indicators.py +++ b/tests/test_app_indicators.py @@ -3,30 +3,31 @@ from hydrology.app.page_modules import indicators -def test_fetch_precip_data_uses_meteostat_precip_mm_fallback(monkeypatch): +def test_fetch_precip_data_uses_shared_climate_result(monkeypatch): calls = [] - def no_daymet(site_id, start_date, end_date, variables): - return None - - def fake_fetch_climate_data(latitude, longitude, start_date, end_date, include_temp, include_precip): + def fake_fetch_climate_result(latitude, longitude, start_date, end_date, site_id, include_temp, include_precip): calls.append( { "latitude": latitude, "longitude": longitude, "start_date": start_date, "end_date": end_date, + "site_id": site_id, "include_temp": include_temp, "include_precip": include_precip, } ) - return pd.DataFrame( - {"Precip_mm": [1.2, 0.0, 4.5]}, - index=pd.date_range("2024-01-01", periods=3, freq="D"), - ) + return { + "data": pd.DataFrame( + {"Precip_mm": [1.2, 0.0, 4.5]}, + index=pd.date_range("2024-01-01", periods=3, freq="D"), + ), + "source": "Meteostat", + "message": "Loaded climate data from the nearest Meteostat station.", + } - monkeypatch.setattr("hydrology.data.hyriver.get_daymet_climate", no_daymet) - monkeypatch.setattr("hydrology.data.climate.fetch_climate_data", fake_fetch_climate_data) + monkeypatch.setattr(indicators, "fetch_climate_cached_result", fake_fetch_climate_result) precip = indicators._fetch_precip_data( "12422500", @@ -41,9 +42,73 @@ def fake_fetch_climate_data(latitude, longitude, start_date, end_date, include_t { "latitude": 47.6593, "longitude": -117.4491, - "start_date": pd.Timestamp("2024-01-01"), - "end_date": pd.Timestamp("2024-01-03"), + "start_date": "2024-01-01", + "end_date": "2024-01-03", + "site_id": "12422500", "include_temp": False, "include_precip": True, } ] + + +def test_fetch_precip_data_result_reports_daymet_source(monkeypatch): + def fake_fetch_climate_result(*args, **kwargs): + return { + "data": pd.DataFrame( + {"Precip_mm": [2.0, 0.0, 1.5]}, + index=pd.date_range("2024-01-01", periods=3, freq="D"), + ), + "source": "Daymet", + "message": "Loaded gridded Daymet climate data for the selected gage.", + } + + monkeypatch.setattr(indicators, "fetch_climate_cached_result", fake_fetch_climate_result) + + result = indicators._fetch_precip_data_result( + "12422500", + "47.6593", + "-117.4491", + "2024-01-01", + "2024-01-03", + ) + + assert result["source"] == "Daymet" + assert result["n_days"] == 3 + assert result["precip"].tolist() == [2.0, 0.0, 1.5] + + +def test_fetch_precip_data_result_reports_unavailable_without_coordinates(monkeypatch): + result = indicators._fetch_precip_data_result( + "12422500", + None, + None, + "2024-01-01", + "2024-01-03", + ) + + assert result["precip"] is None + assert result["source"] == "Unavailable" + assert "missing site coordinates" in result["message"] + + +def test_spi_readiness_rows_summarize_source_and_record_length(): + rows = indicators._spi_readiness_rows( + {"source": "Daymet", "n_days": 3650, "message": "Loaded precipitation from Daymet."} + ) + + assert rows == [ + {"Item": "Precipitation source", "Value": "Daymet"}, + {"Item": "Daily precipitation records", "Value": "3,650"}, + {"Item": "Status", "Value": "Loaded precipitation from Daymet."}, + ] +import pandas as pd + + +def test_summarize_baseflow_methods_returns_dashboard_fields(): + from hydrology.app.page_modules.indicators import _summarize_baseflow_methods + + flow = pd.Series([100, 110, 130, 120, 105, 95, 90, 100], index=pd.date_range("2024-01-01", periods=8)) + + summary = _summarize_baseflow_methods(flow) + + assert {"Lyne-Hollick BFI", "Eckhardt BFI", "Difference", "Agreement"}.issubset(summary) diff --git a/tests/test_app_interpretation.py b/tests/test_app_interpretation.py index 3f92a55..6d36a8a 100644 --- a/tests/test_app_interpretation.py +++ b/tests/test_app_interpretation.py @@ -16,6 +16,7 @@ def test_summarize_flow_context_includes_current_and_record_depth(): assert cards[0].title == "Current Flow" assert cards[0].value.endswith("cfs") assert any(card.title == "Record Depth" and "12.0" in card.value for card in cards) + assert all(getattr(card, "help", "") for card in cards) def test_summarize_flow_context_handles_empty_data(): @@ -42,3 +43,100 @@ def test_describe_standardized_index_explains_dry_and_normal_values(): assert "drier" in dry_body assert normal_label == "Near normal" assert "historical middle range" in normal_body + + +def test_build_plot_analysis_report_covers_core_sections(): + from hydrology.app.interpretation import build_plot_analysis_report + + index = pd.date_range("2015-01-01", periods=365 * 5, freq="D") + # Seasonal signal: higher in spring + q = [] + for ts in index: + base = 50 + 200 * (1 if ts.month in (3, 4, 5) else 0.2) + q.append(base) + df = pd.DataFrame({"Discharge_cfs": q}, index=index) + df["Gage_Height_ft"] = 2 + df["Discharge_cfs"] ** 0.3 / 10 + merged = df.copy() + merged["Precip_mm"] = 1.0 + merged["Temp_C"] = 10.0 + + report = build_plot_analysis_report( + site_id="14018500", + site_desc="WALLA WALLA RIVER NEAR TOUCHET, WA", + plot_keys=["timeseries", "flow_duration", "rating_curve"], + df_q=df, + df_merged=merged, + ) + + assert "14018500" in report + assert "Flow duration" in report + assert "Seasonal pattern" in report + assert "Climate" in report + assert "Plots generated" in report + assert "Metric relevance" in report + assert "Q50" in report or "Median" in report + + +def test_metric_relevance_table_has_core_metrics(): + from hydrology.app.interpretation import metric_relevance_table, metric_keys_for_plots + + rows = metric_relevance_table(["q10", "q90", "mean_flow"]) + assert len(rows) == 3 + assert {r["Metric"] for r in rows} == { + "Q10 (high flow)", + "Q90 (low flow)", + "Mean flow", + } + keys = metric_keys_for_plots(["flow_duration", "rating_curve"]) + assert "rating_r2" in keys + assert "q90" in keys + + +def test_dynamic_metric_relevance_uses_site_hydrology(): + from hydrology.app.interpretation import ( + compute_hydrologic_profile, + dynamic_metric_relevance, + format_metric_relevance_markdown, + metric_help_text, + ) + + index = pd.date_range("2010-01-01", periods=365 * 8, freq="D") + q = [] + for ts in index: + # Flashy spring peaks vs low summer baseflow + if ts.month in (3, 4, 5): + q.append(800 + (ts.day % 20) * 40) + elif ts.month in (7, 8): + q.append(25 + (ts.day % 5)) + else: + q.append(120 + (ts.day % 10) * 5) + df = pd.DataFrame({"Discharge_cfs": q}, index=index) + df["Gage_Height_ft"] = 1.5 + (df["Discharge_cfs"] / 50) ** 0.4 + merged = df.copy() + merged["Precip_mm"] = 0.8 + merged["Temp_C"] = 9.0 + + profile = compute_hydrologic_profile(df, merged) + assert profile["ok"] + assert profile["q10"] > profile["q50"] > profile["q90"] + assert profile["regime"] in {"flashy", "seasonal", "steady"} + + rows = dynamic_metric_relevance( + df, merged, metric_keys=["mean_flow", "q10", "q50", "q90", "spi_sri", "rating_r2"] + ) + assert len(rows) == 6 + why = " ".join(r["Why it matters here"] for r in rows) + assert f"{profile['q50']:,.0f}" in why or "Median" in why or "cfs" in why + assert "This site/period" in rows[0] + assert rows[0]["This site/period"] != "—" + assert any(r["Regime"] == profile["regime_plain"] for r in rows) + + md = format_metric_relevance_markdown( + ["q50", "q90"], df_q=df, df_merged=merged + ) + assert "regime" in md.lower() + assert str(int(profile["q50"])) in md.replace(",", "") or f"{profile['q50']:,.0f}" in md + + tip = metric_help_text("q10", df, merged) + assert "cfs" in tip.lower() or "Q10" in tip + assert tip != "" diff --git a/tests/test_app_navigation_labels.py b/tests/test_app_navigation_labels.py index fe6d8c6..7be3488 100644 --- a/tests/test_app_navigation_labels.py +++ b/tests/test_app_navigation_labels.py @@ -11,10 +11,8 @@ def test_main_navigation_uses_role_specific_labels(): "Stations", "Site Analysis", "Compare Sites", - "Reach Tools", - "Current Check", - "Climate Indicators", - "More Tools", + "Reach Analysis", + "Watershed", ]: assert label in text @@ -23,29 +21,55 @@ def test_main_navigation_uses_role_specific_labels(): ">Analyze<", ">Compare<", ">Monitor<", + "More Tools", + "Current Check", + "Climate Indicators", ]: assert old_label not in text -def test_workflow_tiles_match_primary_navigation_terms(): - text = (ROOT / "hydrology/app/styles.py").read_text(encoding="utf-8") +def test_app_shell_uses_coalesced_page_set(): + text = (ROOT / "hydrology/app/app.py").read_text(encoding="utf-8") - assert '"title": "Stations"' in text - assert '"title": "Site Analysis"' in text - assert '"title": "Compare Sites"' in text - assert '"title": "Current Check"' in text + for title in [ + 'title="Stations"', + 'title="Site Analysis"', + 'title="Compare Sites"', + 'title="Reach Analysis"', + 'title="Watershed"', + ]: + assert title in text + for removed in [ + 'title="Alerts"', + 'title="Indicators"', + 'title="Advanced"', + "render_workflow_strip()", + ]: + assert removed not in text -def test_workflow_tiles_cover_all_navigation_roles(): - text = (ROOT / "hydrology/app/styles.py").read_text(encoding="utf-8") - for label in [ - '"title": "Stations"', - '"title": "Site Analysis"', - '"title": "Compare Sites"', - '"title": "Reach Tools"', - '"title": "Current Check"', - '"title": "Climate Indicators"', - '"title": "More Tools"', - ]: - assert label in text +def test_compare_page_owns_multisite_relationships(): + text = (ROOT / "hydrology/app/page_modules/comparisons.py").read_text(encoding="utf-8") + + assert "Site Relationships" in text + assert "_multisite_analysis" in text + + +def test_site_analysis_owns_indicators_without_autoloading(): + text = (ROOT / "hydrology/app/page_modules/single_analysis.py").read_text(encoding="utf-8") + + assert "Drought & Baseflow Indicators" in text + assert "Load Indicators" in text + assert "render_site_indicators" in text + assert "indicator_loaded_key" in text + assert "indicator_button_key" in text + assert "st.session_state[indicator_button_key]" not in text + + +def test_stations_owns_current_conditions_check(): + text = (ROOT / "hydrology/app/page_modules/overview.py").read_text(encoding="utf-8") + + assert "Current Conditions Check" in text + assert "render_site_current_check" in text + assert "alerts?site=" not in text diff --git a/tests/test_app_reach_analysis.py b/tests/test_app_reach_analysis.py new file mode 100644 index 0000000..029b73e --- /dev/null +++ b/tests/test_app_reach_analysis.py @@ -0,0 +1,490 @@ +import pandas as pd +import inspect +import hydrology.app.page_modules.reach_analysis as reach_analysis_module + +from hydrology.app.page_modules.reach_analysis import ( + _build_reach_interpretation, + _build_recommended_reach_pairs, + _build_reach_summary_row, + _build_reach_candidate_options, + _candidate_index_for_site, + _candidate_label_for_site, + _default_candidate_label, + _estimate_reach_km, + _filter_related_sites_to_inventory, + _format_reach_chain, + _format_related_site_rows, + _flowline_distance_km, + _flowline_style, + _clip_flowlines_between_gages, + _leaflet_fit_bounds_script, + _map_bounds_for_reach, + _pair_key, + _cycle_pair_key, + _pair_label_for_key, + _reach_map_component_key, + _resolve_selected_pair_key, + _resolve_reach_km, + _selectbox_kwargs_for_state, + _selected_candidate_site_id, +) + + +def test_build_reach_summary_row_formats_gain_loss_for_dashboard(): + index = pd.date_range("2021-08-01", periods=10, freq="D") + upstream = pd.Series([100] * 10, index=index) + downstream = pd.Series([85] * 10, index=index) + + row = _build_reach_summary_row("up", "down", upstream, downstream, reach_km=5) + + assert row["Reach"] == "up -> down" + assert row["Class"] == "losing" + assert row["Median gain/loss"] == "-15 cfs" + assert row["Gain/loss per km"] == "-3.0 cfs/km" + assert row["Confidence"] == "high" + + +def test_build_reach_summary_row_explains_missing_reach_length(): + index = pd.date_range("2021-08-01", periods=10, freq="D") + upstream = pd.Series([100] * 10, index=index) + downstream = pd.Series([115] * 10, index=index) + + row = _build_reach_summary_row("up", "down", upstream, downstream) + + assert row["Gain/loss per km"] == "Add reach length" + + +def test_format_reach_chain_shows_upstream_to_downstream_order(): + rows = _format_reach_chain(["headwater", "mid", "lower"]) + + assert rows == [ + {"Reach": "headwater -> mid", "Order": 1}, + {"Reach": "mid -> lower", "Order": 2}, + ] + + +def test_estimate_reach_km_from_navigation_distance_difference(): + related_sites = [ + {"site_id": "up", "direction": "upstream", "distance_km": 12.0}, + {"site_id": "down", "direction": "downstream", "distance_km": 8.0}, + ] + + assert _estimate_reach_km("up", "origin", related_sites, "origin") == 12.0 + assert _estimate_reach_km("origin", "down", related_sites, "origin") == 8.0 + assert _estimate_reach_km("up", "down", related_sites, "origin") == 20.0 + + +def test_resolve_reach_km_prefers_estimated_network_length(): + assert _resolve_reach_km(12.3, 20.0) == 12.3 + + +def test_resolve_reach_km_uses_manual_override_when_estimate_missing(): + assert _resolve_reach_km(None, 20.0) == 20.0 + + +def test_resolve_reach_km_allows_missing_length(): + assert _resolve_reach_km(None, 0.0) is None + + +def test_flowline_distance_uses_reach_length_with_buffer(): + assert _flowline_distance_km(12.0, 75.0) == 18.0 + + +def test_flowline_distance_falls_back_to_search_distance(): + assert _flowline_distance_km(None, 75.0) == 75.0 + + +def test_flowline_style_highlights_selected_network(): + assert _flowline_style(selected=True)["weight"] > _flowline_style(selected=False)["weight"] + + +def test_map_bounds_focuses_on_selected_gage_markers_not_flowline_extent(): + class FakeFlowlines: + total_bounds = [-125.0, 40.0, -110.0, 50.0] + empty = False + + bounds = _map_bounds_for_reach( + FakeFlowlines(), + context_flowlines=object(), + upstream_lat=10.0, + upstream_lon=20.0, + downstream_lat=11.0, + downstream_lon=21.0, + ) + + assert bounds == [[9.75, 19.75], [11.25, 21.25]] + + +def test_map_bounds_pads_close_gage_markers(): + bounds = _map_bounds_for_reach( + selected_flowlines=None, + context_flowlines=None, + upstream_lat=10.0, + upstream_lon=20.0, + downstream_lat=10.001, + downstream_lon=20.001, + ) + + assert bounds == [[9.99, 19.99], [10.011, 20.011]] + + +def test_build_reach_interpretation_summarizes_gaining_reach(): + row = { + "Reach": "up -> down", + "Class": "gaining", + "Median gain/loss": "342 cfs", + "Low-flow gain/loss": "339 cfs", + "Gain/loss per km": "12.0 cfs/km", + "Confidence": "moderate", + } + + summary = _build_reach_interpretation(row, reach_km=28.5, length_source="network") + + assert summary["Finding"] == "Gaining reach" + assert "downstream flow is higher" in summary["Interpretation"] + assert summary["Reach length"] == "28.5 km, network inferred" + assert summary["Review"] == "Screening result; check tributaries, diversions, withdrawals, and data overlap." + + +def test_build_reach_interpretation_explains_missing_length(): + row = { + "Reach": "up -> down", + "Class": "insufficient_data", + "Median gain/loss": "N/A", + "Low-flow gain/loss": "N/A", + "Gain/loss per km": "Add reach length", + "Confidence": "none", + } + + summary = _build_reach_interpretation(row, reach_km=None, length_source="missing") + + assert summary["Finding"] == "Not enough paired data" + assert summary["Reach length"] == "Not inferred" + assert "cfs/km is unavailable" in summary["Interpretation"] + + +def test_format_related_site_rows_makes_station_choices_legible(): + rows = _format_related_site_rows( + "anchor", + [ + { + "site_id": "up", + "name": "Upstream mainstem gage", + "direction": "upstream", + "distance_km": 29.4, + }, + { + "site_id": "trib", + "name": "Tributary gage", + "direction": "upstream", + "navigation_mode": "upstream_trib", + "distance_km": 12.0, + }, + { + "site_id": "down", + "name": "Downstream gage", + "direction": "downstream", + "distance_km": 18.1, + }, + ], + ) + + assert rows[0]["Station"] == "anchor" + assert rows[0]["Position"] == "Anchor" + assert rows[1]["Position"] == "Upstream" + assert rows[1]["Distance from anchor"] == "29.4 km" + assert rows[2]["Position"] == "Tributary" + assert rows[3]["Position"] == "Downstream" + + +def test_build_reach_candidate_options_groups_both_directions(): + candidates = _build_reach_candidate_options( + "anchor", + "Anchor gage", + [ + { + "site_id": "up", + "name": "Upstream gage", + "direction": "upstream", + "distance_km": 5.2, + }, + { + "site_id": "down", + "name": "Downstream gage", + "direction": "downstream", + "distance_km": 7.8, + }, + ], + ) + + labels = [candidate["label"] for candidate in candidates] + + assert labels[0].startswith("Anchor | anchor") + assert labels[1].startswith("Upstream | up") + assert labels[2].startswith("Downstream | down") + assert candidates[1]["site_id"] == "up" + assert candidates[2]["site_id"] == "down" + + +def test_filter_related_sites_to_inventory_keeps_only_processable_gages(): + inventory = pd.DataFrame( + { + "site_id": ["anchor", "usable"], + "description": ["Anchor gage", "Usable related gage"], + } + ) + related_sites = [ + {"site_id": "usable", "direction": "upstream", "distance_km": 4.0}, + {"site_id": "outside", "direction": "downstream", "distance_km": 8.0}, + ] + + filtered, omitted = _filter_related_sites_to_inventory("anchor", related_sites, inventory) + + assert [site["site_id"] for site in filtered] == ["usable"] + assert omitted == ["outside"] + + +def test_filter_related_sites_to_inventory_keeps_anchor_even_if_not_related(): + inventory = pd.DataFrame({"site_id": ["anchor"]}) + + filtered, omitted = _filter_related_sites_to_inventory("anchor", [], inventory) + + assert filtered == [] + assert omitted == [] + + +def test_reach_map_component_key_changes_with_selected_pair_and_bounds(): + first = _reach_map_component_key("up", "down", [[10.0, 20.0], [11.0, 21.0]]) + second = _reach_map_component_key("up", "other", [[10.0, 20.0], [11.0, 21.0]]) + + assert first != second + assert first.startswith("reach_map_up_down_") + + +def test_selectbox_kwargs_omit_index_when_session_state_has_widget_key(): + kwargs = _selectbox_kwargs_for_state("reach_upstream_choice", 2, {"reach_upstream_choice": "label"}) + + assert kwargs == {"key": "reach_upstream_choice"} + + +def test_selectbox_kwargs_include_index_for_initial_render(): + kwargs = _selectbox_kwargs_for_state("reach_upstream_choice", 2, {}) + + assert kwargs == {"key": "reach_upstream_choice", "index": 2} + + +def test_candidate_index_for_site_prefers_selected_site(): + candidates = [ + {"site_id": "anchor", "position": "Anchor"}, + {"site_id": "up", "position": "Upstream"}, + {"site_id": "down", "position": "Downstream"}, + ] + + assert _candidate_index_for_site(candidates, "down", {"Upstream"}) == 2 + + +def test_candidate_label_for_site_returns_matching_dropdown_label(): + candidates = [ + {"site_id": "anchor", "label": "Anchor | anchor | 0.0 km | Anchor"}, + {"site_id": "up", "label": "Upstream | up | 5.0 km | Upstream"}, + ] + + assert _candidate_label_for_site(candidates, "up") == "Upstream | up | 5.0 km | Upstream" + + +def test_default_candidate_label_uses_preferred_site_before_role_fallback(): + candidates = [ + {"site_id": "anchor", "label": "Anchor label", "position": "Anchor"}, + {"site_id": "up", "label": "Upstream label", "position": "Upstream"}, + {"site_id": "down", "label": "Downstream label", "position": "Downstream"}, + ] + + assert _default_candidate_label(candidates, "down", {"Upstream"}) == "Downstream label" + + +def test_default_candidate_label_falls_back_to_role_label(): + candidates = [ + {"site_id": "anchor", "label": "Anchor label", "position": "Anchor"}, + {"site_id": "up", "label": "Upstream label", "position": "Upstream"}, + ] + + assert _default_candidate_label(candidates, None, {"Upstream"}) == "Upstream label" + + +def test_candidate_index_for_site_falls_back_to_role(): + candidates = [ + {"site_id": "anchor", "position": "Anchor"}, + {"site_id": "up", "position": "Upstream"}, + {"site_id": "down", "position": "Downstream"}, + ] + + assert _candidate_index_for_site(candidates, None, {"Upstream"}) == 1 + + +def test_selected_candidate_site_id_reads_single_selected_table_row(): + candidate_rows = [ + {"Station": "anchor"}, + {"Station": "up"}, + {"Station": "down"}, + ] + selection_state = {"selection": {"rows": [2]}} + + assert _selected_candidate_site_id(candidate_rows, selection_state) == "down" + + +def test_build_recommended_reach_pairs_prefers_mainstem_pairs(): + candidates = [ + {"site_id": "anchor", "position": "Anchor", "distance_km": 0.0, "label": "Anchor | anchor | 0.0 km | Anchor"}, + {"site_id": "up", "position": "Upstream", "distance_km": 5.0, "label": "Upstream | up | 5.0 km | Upstream"}, + {"site_id": "trib", "position": "Tributary", "distance_km": 3.0, "label": "Tributary | trib | 3.0 km | Tributary"}, + {"site_id": "down", "position": "Downstream", "distance_km": 8.0, "label": "Downstream | down | 8.0 km | Downstream"}, + ] + + pairs = _build_recommended_reach_pairs("anchor", candidates, max_pairs=5) + + assert pairs[0]["upstream_id"] == "up" + assert pairs[0]["downstream_id"] == "anchor" + assert pairs[1]["upstream_id"] == "anchor" + assert pairs[1]["downstream_id"] == "down" + assert all(pair["upstream_id"] != pair["downstream_id"] for pair in pairs) + assert any(pair["kind"] == "tributary context" for pair in pairs) + + +def test_pair_key_is_stable_and_readable(): + assert _pair_key("12419000", "12422000") == "12419000__12422000" + + +def test_pair_label_for_key_returns_matching_label(): + pairs = [ + {"key": "a__b", "label": "A to B"}, + {"key": "b__c", "label": "B to C"}, + ] + + assert _pair_label_for_key(pairs, "b__c") == "B to C" + + +def test_cycle_pair_key_steps_forward_and_wraps(): + pairs = [{"key": "a"}, {"key": "b"}, {"key": "c"}] + + assert _cycle_pair_key(pairs, "a", 1) == "b" + assert _cycle_pair_key(pairs, "c", 1) == "a" + + +def test_cycle_pair_key_steps_backward_and_wraps(): + pairs = [{"key": "a"}, {"key": "b"}, {"key": "c"}] + + assert _cycle_pair_key(pairs, "a", -1) == "c" + + +def test_resolve_selected_pair_key_keeps_valid_existing_selection(): + pairs = [ + {"key": "up__anchor", "upstream_id": "up", "downstream_id": "anchor"}, + {"key": "anchor__down", "upstream_id": "anchor", "downstream_id": "down"}, + ] + session_state = {"reach_selected_pair_key": "anchor__down"} + + assert _resolve_selected_pair_key(pairs, session_state) == "anchor__down" + + +def test_resolve_selected_pair_key_falls_back_to_first_pair(): + pairs = [{"key": "up__anchor", "upstream_id": "up", "downstream_id": "anchor"}] + session_state = {"reach_selected_pair_key": "missing__pair"} + + assert _resolve_selected_pair_key(pairs, session_state) == "up__anchor" + + +def test_resolve_selected_pair_key_handles_no_pairs(): + assert _resolve_selected_pair_key([], {}) is None + + +def test_leaflet_fit_bounds_script_targets_selected_bounds(): + script = _leaflet_fit_bounds_script([[47.0, -118.0], [48.0, -117.0]]) + + assert "fitBounds" in script + assert "[[47.0, -118.0], [48.0, -117.0]]" in script + assert "paddingTopLeft" in script + assert "paddingBottomRight" in script + + +def test_clip_flowlines_between_gages_limits_highlight_to_selected_points(): + import geopandas as gpd + from shapely.geometry import LineString + + flowlines = gpd.GeoDataFrame( + {"name": ["mainstem"]}, + geometry=[LineString([(0, 0), (10, 0)])], + crs="EPSG:3857", + ) + + clipped = _clip_flowlines_between_gages( + flowlines, + upstream_lat=0, + upstream_lon=3, + downstream_lat=0, + downstream_lon=7, + ) + + assert clipped is not None + assert round(clipped.geometry.iloc[0].length, 6) == 4 + assert list(clipped.geometry.iloc[0].coords) == [(3.0, 0.0), (7.0, 0.0)] + + +def test_clip_flowlines_between_gages_traces_connected_mainline_segments(): + import geopandas as gpd + from shapely.geometry import LineString + + flowlines = gpd.GeoDataFrame( + {"name": ["lower", "middle", "upper"]}, + geometry=[ + LineString([(0, 0), (4, 0)]), + LineString([(4, 0), (4, 3)]), + LineString([(4, 3), (8, 3)]), + ], + crs="EPSG:3857", + ) + + clipped = _clip_flowlines_between_gages( + flowlines, + upstream_lat=3, + upstream_lon=7, + downstream_lat=0, + downstream_lon=1, + ) + + assert clipped is not None + coords = list(clipped.geometry.iloc[0].coords) + assert coords == [(7.0, 3.0), (4.0, 3.0), (4.0, 0.0), (1.0, 0.0)] + assert round(clipped.geometry.iloc[0].length, 6) == 9.0 + + +def test_build_recommended_reach_pairs_labels_include_context(): + candidates = [ + {"site_id": "anchor", "position": "Anchor", "distance_km": 0.0, "label": "Anchor | anchor | 0.0 km | Anchor gage"}, + {"site_id": "up", "position": "Upstream", "distance_km": 5.25, "label": "Upstream | up | 5.2 km | Upper River near Town"}, + ] + + pairs = _build_recommended_reach_pairs("anchor", candidates) + + assert pairs[0]["label"] == "Upstream: up -> anchor | 5.2 km | Upper River near Town" + + +def test_reach_page_source_uses_gage_terminology(): + source = inspect.getsource(reach_analysis_module.show) + forbidden = "gau" + "ge" + + assert forbidden not in source.lower() + assert "gage" in source.lower() + + +def test_reach_page_source_does_not_bury_map_in_expander(): + source = inspect.getsource(reach_analysis_module.show) + + assert 'st.expander("Reach Map"' not in source + + +def test_reach_map_uses_full_flowline_context_for_selected_path(): + source = inspect.getsource(reach_analysis_module._render_reach_map) + + assert "get_navigation_flowlines" not in source + assert "_clip_flowlines_between_gages(\n flowlines," in source + assert "context only" in source diff --git a/tests/test_app_shared_conditions.py b/tests/test_app_shared_conditions.py index 3f49bfa..c740f13 100644 --- a/tests/test_app_shared_conditions.py +++ b/tests/test_app_shared_conditions.py @@ -79,8 +79,9 @@ def test_normalize_climate_columns_converts_daymet_names(): assert str(climate.index.tz) == "UTC" -def test_fetch_climate_cached_prefers_station_data_before_daymet(monkeypatch): +def test_fetch_climate_cached_prefers_daymet_for_site_climate(monkeypatch): shared.fetch_climate_cached.clear() + shared.fetch_climate_cached_result.clear() calls = [] station = pd.DataFrame( @@ -94,7 +95,10 @@ def fake_station(*args, **kwargs): def fake_daymet(*args, **kwargs): calls.append("daymet") - return pd.DataFrame() + return pd.DataFrame( + {"precip_mm": [1.0], "tmin_c": [2.0], "tmax_c": [8.0]}, + index=pd.date_range("2024-01-01", periods=1, freq="D"), + ) monkeypatch.setattr(shared, "fetch_climate_data", fake_station) monkeypatch.setattr("hydrology.data.hyriver.get_daymet_climate", fake_daymet) @@ -102,4 +106,34 @@ def fake_daymet(*args, **kwargs): climate = shared.fetch_climate_cached(47.0, -117.0, "2024-01-01", "2024-01-01", "12422500") assert climate["Temp_C"].tolist() == [5.0] - assert calls == ["station"] + assert climate["Precip_mm"].tolist() == [1.0] + assert calls == ["daymet"] + + +def test_fetch_climate_cached_falls_back_to_station_data(monkeypatch): + shared.fetch_climate_cached.clear() + shared.fetch_climate_cached_result.clear() + calls = [] + + station = pd.DataFrame( + {"Temp_C": [6.0], "Precip_mm": [0.4]}, + index=pd.date_range("2024-01-01", periods=1, freq="D"), + ) + + def fake_station(*args, **kwargs): + calls.append("station") + return station + + def fake_daymet(*args, **kwargs): + calls.append("daymet") + return None + + monkeypatch.setattr(shared, "fetch_climate_data", fake_station) + monkeypatch.setattr("hydrology.data.hyriver.get_daymet_climate", fake_daymet) + + result = shared.fetch_climate_cached_result(47.0, -117.0, "2024-01-01", "2024-01-01", "12422500") + + assert result["source"] == "Meteostat" + assert result["data"]["Temp_C"].tolist() == [6.0] + assert result["data"]["Precip_mm"].tolist() == [0.4] + assert calls == ["daymet", "station"] diff --git a/tests/test_app_visual_shell.py b/tests/test_app_visual_shell.py index c2a6edf..5c5d730 100644 --- a/tests/test_app_visual_shell.py +++ b/tests/test_app_visual_shell.py @@ -20,3 +20,42 @@ def test_visual_system_avoids_sidebar_first_language(): assert "Search in the sidebar" not in text assert "sidebar is not the primary" in text + + +def test_styles_include_tile_hover_tooltips(): + text = (ROOT / "hydrology/app/styles.py").read_text(encoding="utf-8") + assert ".tile-tip" in text + assert "def _tile_tip_html" in text + assert "section: str = \"all\"" in text or 'section: str = "all"' in text + + +def test_styles_include_mobile_touch_and_column_stacking(): + text = (ROOT / "hydrology/app/styles.py").read_text(encoding="utf-8") + assert "@media (max-width: 768px)" in text + assert "@media (hover: none) and (pointer: coarse)" in text + assert "tabindex=" in text or 'tabindex="0"' in text + # Columns stack on tablet/phone so rating workshop isn't side-squeezed + assert 'min-width: 100% !important' in text + assert "stHorizontalBlock" in text + + +def test_styles_include_premium_motion_and_reduced_motion_guard(): + """New fluid/animated premium polish (inspired by modern award-winning web UIs) + must be present in the CSS system and guarded for accessibility. + """ + text = (ROOT / "hydrology/app/styles.py").read_text(encoding="utf-8") + + for token in [ + "@keyframes fadeInUp", + "@keyframes cardPop", + "@keyframes subtlePulse", + "animation: fadeInUp", + "animation: cardPop", + "prefers-reduced-motion: reduce", + "translateY(-2px)", + "will-change: transform", + ]: + assert token in text, f"Missing premium motion token: {token}" + + # Inspiration comment / docstring reference present + assert "x.com/twetsfyp/status/2065283731833651709" in text or "award-winning" in text diff --git a/tests/test_architecture_docs.py b/tests/test_architecture_docs.py new file mode 100644 index 0000000..82c1374 --- /dev/null +++ b/tests/test_architecture_docs.py @@ -0,0 +1,20 @@ +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] + + +def test_architecture_doc_names_current_workflows_and_groundwater_rules(): + text = (ROOT / "docs/architecture.md").read_text(encoding="utf-8") + + for workflow in ["Stations", "Site Analysis", "Compare Sites", "Reach Analysis", "Watershed"]: + assert workflow in text + + for rule in [ + "USGS groundwater field measurements", + "Washington Ecology EIM", + "Washington Ecology well logs", + "Drop private fields", + "Do not fetch groundwater data automatically", + ]: + assert rule in text diff --git a/tests/test_baseflow.py b/tests/test_baseflow.py new file mode 100644 index 0000000..0505606 --- /dev/null +++ b/tests/test_baseflow.py @@ -0,0 +1,48 @@ +import numpy as np +import pandas as pd + +from hydrology.analysis.baseflow import ( + BaseflowResult, + compare_baseflow_methods, + eckhardt_filter, + lyne_hollick_filter, +) + + +def _daily_flow(values): + return pd.Series(values, index=pd.date_range("2020-01-01", periods=len(values), freq="D")) + + +def test_lyne_hollick_returns_components_bounded_by_total_flow(): + flow = _daily_flow([100, 120, 180, 140, 110, 90, 95, 105, 130, 115]) + + result = lyne_hollick_filter(flow, alpha=0.925, passes=3) + + assert isinstance(result, BaseflowResult) + assert list(result.components.columns) == ["total_flow", "baseflow", "quickflow"] + assert (result.components["baseflow"] >= 0).all() + assert (result.components["baseflow"] <= result.components["total_flow"]).all() + assert 0 <= result.bfi <= 1 + assert result.method == "lyne_hollick" + + +def test_eckhardt_filter_responds_to_bfi_max_and_stays_bounded(): + flow = _daily_flow([100, 110, 130, 125, 115, 105, 95, 100, 108, 102]) + + conservative = eckhardt_filter(flow, alpha=0.98, bfi_max=0.50) + permissive = eckhardt_filter(flow, alpha=0.98, bfi_max=0.80) + + assert (conservative.components["baseflow"] <= conservative.components["total_flow"]).all() + assert (permissive.components["baseflow"] <= permissive.components["total_flow"]).all() + assert permissive.bfi >= conservative.bfi + assert conservative.method == "eckhardt" + + +def test_compare_baseflow_methods_flags_large_disagreement(): + flow = _daily_flow(np.r_[np.repeat(100.0, 20), 500.0, np.repeat(90.0, 20)]) + + comparison = compare_baseflow_methods(flow) + + assert {"lyne_hollick_bfi", "eckhardt_bfi", "bfi_difference", "agreement"}.issubset(comparison) + assert comparison["agreement"] in {"strong", "moderate", "weak"} + assert comparison["bfi_difference"] >= 0 diff --git a/tests/test_case_runner.py b/tests/test_case_runner.py new file mode 100644 index 0000000..f409715 --- /dev/null +++ b/tests/test_case_runner.py @@ -0,0 +1,148 @@ +from pathlib import Path + +import yaml + +from scripts.run_case_study import run_case_study + + +def test_run_case_study_writes_summary(tmp_path: Path): + case_dir = tmp_path / "case" + case_dir.mkdir() + config_path = case_dir / "case.yml" + config_path.write_text( + yaml.safe_dump( + { + "case": "example_case", + "analyses": ["baseflow", "signatures"], + } + ), + encoding="utf-8", + ) + + summary_path = run_case_study(config_path) + + assert summary_path == case_dir / "outputs" / "run_summary.md" + assert "example_case" in summary_path.read_text(encoding="utf-8") + + +def test_run_case_study_writes_baseflow_and_signature_outputs(tmp_path: Path): + case_dir = tmp_path / "case" + case_dir.mkdir() + flow_path = case_dir / "daily_flow.csv" + flow_path.write_text( + "date,discharge_cfs\n" + "2020-01-01,100\n" + "2020-01-02,120\n" + "2020-01-03,140\n" + "2020-01-04,110\n" + "2020-01-05,90\n" + "2020-01-06,95\n", + encoding="utf-8", + ) + config_path = case_dir / "case.yml" + config_path.write_text( + yaml.safe_dump( + { + "case": "flow_case", + "input_daily_flow_csv": str(flow_path), + "analyses": ["baseflow", "signatures"], + } + ), + encoding="utf-8", + ) + + run_case_study(config_path) + + assert (case_dir / "outputs" / "baseflow_components.csv").exists() + assert (case_dir / "outputs" / "signatures.csv").exists() + + +def test_run_case_study_writes_validation_summary(tmp_path: Path): + case_dir = tmp_path / "case" + case_dir.mkdir() + flow_path = case_dir / "daily_flow.csv" + flow_path.write_text( + "date,discharge_cfs\n" + "2020-01-01,100\n" + "2020-01-02,120\n" + "2020-01-03,140\n" + "2020-01-04,110\n" + "2020-01-05,90\n" + "2020-01-06,95\n", + encoding="utf-8", + ) + config_path = case_dir / "case.yml" + config_path.write_text( + yaml.safe_dump( + { + "case": "validated_flow_case", + "input_daily_flow_csv": str(flow_path), + "analyses": ["baseflow", "signatures"], + "validation": [ + {"metric": "baseflow_index_lh", "lower": 0.0, "upper": 1.0}, + {"metric": "n_days", "expected": 6, "tolerance": 0.0}, + ], + } + ), + encoding="utf-8", + ) + + run_case_study(config_path) + + summary = case_dir / "outputs" / "validation_summary.csv" + assert summary.exists() + text = summary.read_text(encoding="utf-8") + assert "baseflow_index_lh" in text + assert "n_days" in text + assert "PASS" in text + + +def test_run_case_study_fetches_daily_flow_from_site_config(tmp_path: Path, monkeypatch): + import pandas as pd + import scripts.run_case_study as runner + + def fake_fetch(site_id, start_date, end_date): + assert site_id == "12345678" + assert start_date == "2020-01-01" + assert end_date == "2020-01-03" + return pd.DataFrame( + {"Discharge_cfs": [100.0, 110.0, 120.0]}, + index=pd.date_range("2020-01-01", periods=3, freq="D"), + ) + + monkeypatch.setattr(runner, "fetch_discharge_data", fake_fetch) + case_dir = tmp_path / "case" + case_dir.mkdir() + config_path = case_dir / "case.yml" + config_path.write_text( + yaml.safe_dump( + { + "case": "fetched_flow_case", + "site_ids": ["12345678"], + "start_date": "2020-01-01", + "end_date": "2020-01-03", + "analyses": ["baseflow"], + "validation": [{"metric": "n_days", "expected": 3, "tolerance": 0.0}], + } + ), + encoding="utf-8", + ) + + run_case_study(config_path) + + assert (case_dir / "outputs" / "baseflow_components.csv").exists() + validation = (case_dir / "outputs" / "validation_summary.csv").read_text(encoding="utf-8") + assert "n_days" in validation + assert "PASS" in validation + + +def test_pnw_case_configs_are_parseable(): + cases = [ + Path("docs/cases/pnw_baseflow_signatures/case.yml"), + Path("docs/cases/spokane_reach_gain_loss/case.yml"), + ] + + for case_path in cases: + config = yaml.safe_load(case_path.read_text(encoding="utf-8")) + assert config["case"] + assert config["analyses"] diff --git a/tests/test_changepoints.py b/tests/test_changepoints.py new file mode 100644 index 0000000..4d63311 --- /dev/null +++ b/tests/test_changepoints.py @@ -0,0 +1,26 @@ +import pandas as pd + +from hydrology.analysis.changepoints import pettitt_test +from hydrology.analysis.trends import mann_kendall_test + + +def test_pettitt_detects_known_step_change(): + index = pd.Index(range(1900, 1940), name="year") + values = pd.Series([10] * 20 + [30] * 20, index=index) + + result = pettitt_test(values) + + assert result["change_index"] in {19, 20} + assert result["change_point"] in {1919, 1920} + assert result["p_value"] < 0.05 + assert result["mean_before"] < result["mean_after"] + + +def test_mann_kendall_exposes_sens_slope_when_available(): + values = pd.Series([1, 2, 3, 4, 5, 6]) + + result = mann_kendall_test(values) + + if result is not None: + assert "sens_slope" in result + assert result["sens_slope"] > 0 diff --git a/tests/test_frequency.py b/tests/test_frequency.py new file mode 100644 index 0000000..c770cda --- /dev/null +++ b/tests/test_frequency.py @@ -0,0 +1,51 @@ +import numpy as np + +from hydrology.analysis.frequency import estimate_return_periods, flood_frequency_diagnostics + + +def test_estimate_return_periods_is_reproducible_with_seed(): + peaks = np.array([100, 120, 130, 150, 180, 220, 260, 300, 360, 420, 500, 620]) + + first = estimate_return_periods(peaks, periods=[10, 50], distribution="lp3", random_seed=42) + second = estimate_return_periods(peaks, periods=[10, 50], distribution="lp3", random_seed=42) + + assert first[["lower_ci", "upper_ci"]].equals(second[["lower_ci", "upper_ci"]]) + + +def test_flood_frequency_diagnostics_returns_plotting_table(): + peaks = np.array([100, 120, 130, 150, 180, 220, 260, 300, 360, 420, 500, 620]) + + diagnostics = flood_frequency_diagnostics(peaks, distribution="lp3") + + expected = {"observed_flow_cfs", "fitted_flow_cfs", "exceedance_prob", "return_period"} + assert expected.issubset(diagnostics.columns) + assert len(diagnostics) == len(peaks) + + +def test_single_analysis_imports_frequency_diagnostics(): + from hydrology.app.page_modules.single_analysis import _format_frequency_diagnostics + + diagnostics = flood_frequency_diagnostics( + np.array([100, 120, 130, 150, 180, 220, 260, 300, 360, 420, 500, 620]), + distribution="lp3", + ) + + formatted = _format_frequency_diagnostics(diagnostics) + + assert {"Observed flow", "Fitted flow", "Return period"}.issubset(formatted.columns) + + +def test_interactive_return_period_uses_readable_linear_x_axis(): + from hydrology.analysis.frequency import fit_flood_frequency, get_plotting_positions + from hydrology.visualization.interactive import interactive_return_period + + peaks = np.array([100, 120, 130, 150, 180, 220, 260, 300, 360, 420, 500, 620]) + observed = get_plotting_positions(peaks) + fits = fit_flood_frequency(peaks) + rp_table = estimate_return_periods(peaks, periods=[2, 5, 10, 25, 50, 100], random_seed=42) + + fig = interactive_return_period(observed, fits, rp_table) + + assert fig.layout.xaxis.type in (None, "linear") + assert fig.layout.yaxis.type == "log" + assert list(fig.layout.xaxis.tickvals) == [2, 5, 10, 25, 50, 100] diff --git a/tests/test_groundwater.py b/tests/test_groundwater.py new file mode 100644 index 0000000..8ba9e7b --- /dev/null +++ b/tests/test_groundwater.py @@ -0,0 +1,133 @@ +"""Groundwater data boundary tests.""" + +from __future__ import annotations + +import sys +import types + +import pandas as pd +import pytest + +from hydrology.data.groundwater import ( + DEFAULT_GROUNDWATER_PARAMETER_CODES, + SAFE_GROUNDWATER_COLUMNS, + fetch_usgs_groundwater_measurements, + normalize_usgs_groundwater_measurements, +) + + +def test_normalize_usgs_groundwater_measurements_drops_private_fields_and_marks_depth_eligible(): + raw = pd.DataFrame( + [ + { + "monitoring_location_id": "USGS-12422500", + "time": "2024-01-15T00:00:00Z", + "parameter_code": "72019", + "parameter_name": "Depth to water level, feet below land surface", + "value": "12.4", + "unit_of_measure": "ft", + "vertical_datum": "NAVD88", + "approval_status": "Approved", + "latitude": 47.0, + "longitude": -117.0, + "owner_name": "private", + "address": "private", + "parcel": "private", + } + ] + ) + + normalized = normalize_usgs_groundwater_measurements(raw) + row = normalized.iloc[0] + + assert list(normalized.columns) == SAFE_GROUNDWATER_COLUMNS + assert "owner_name" not in normalized.columns + assert "address" not in normalized.columns + assert "parcel" not in normalized.columns + assert row["site_id"] == "12422500" + assert row["depth_to_water_ft"] == pytest.approx(12.4) + assert pd.isna(row["water_level_ft"]) + assert bool(row["analysis_eligible"]) is True + assert row["data_use"] == "analysis" + + +def test_normalize_usgs_groundwater_measurements_maps_water_level_elevation(): + raw = pd.DataFrame( + [ + { + "monitoringLocationId": "USGS-12422500", + "dateTime": "2024-02-15", + "parameterCode": "62611", + "value": "2011.2", + } + ] + ) + + normalized = normalize_usgs_groundwater_measurements(raw) + row = normalized.iloc[0] + + assert row["water_level_ft"] == pytest.approx(2011.2) + assert pd.isna(row["depth_to_water_ft"]) + assert bool(row["analysis_eligible"]) is True + + +def test_normalize_usgs_groundwater_measurements_marks_missing_measurement_limited(): + raw = pd.DataFrame( + [ + { + "monitoring_location_id": "USGS-12422500", + "time": None, + "parameter_code": "00060", + "value": "100", + } + ] + ) + + normalized = normalize_usgs_groundwater_measurements(raw) + row = normalized.iloc[0] + + assert bool(row["analysis_eligible"]) is False + assert row["data_use"] == "limited" + assert "missing measurement date" in row["notes"] + assert "missing supported groundwater measurement" in row["notes"] + + +def test_fetch_usgs_groundwater_measurements_uses_waterdata_client(monkeypatch): + calls = {} + + def fake_get_field_measurements(**params): + calls.update(params) + raw = pd.DataFrame( + [ + { + "monitoring_location_id": "USGS-12422500", + "time": "2024-01-15", + "parameter_code": "72019", + "value": "12.4", + } + ] + ) + return raw, {"ok": True} + + fake_dataretrieval = types.SimpleNamespace( + waterdata=types.SimpleNamespace(get_field_measurements=fake_get_field_measurements) + ) + monkeypatch.setitem(sys.modules, "dataretrieval", fake_dataretrieval) + + normalized = fetch_usgs_groundwater_measurements( + ["12422500"], + start_date="2024-01-01", + end_date="2024-02-01", + ) + + assert calls["monitoring_location_id"] == ["USGS-12422500"] + assert calls["parameter_code"] == DEFAULT_GROUNDWATER_PARAMETER_CODES + assert calls["time"] == "2024-01-01/2024-02-01" + assert calls["limit"] == 50000 + assert calls["skip_geometry"] is False + assert normalized.iloc[0]["site_id"] == "12422500" + + +def test_fetch_usgs_groundwater_measurements_requires_site_or_bbox(): + with pytest.raises(ValueError, match="site_ids or bbox"): + fetch_usgs_groundwater_measurements() diff --git a/tests/test_nwm.py b/tests/test_nwm.py new file mode 100644 index 0000000..72b9ac0 --- /dev/null +++ b/tests/test_nwm.py @@ -0,0 +1,26 @@ +import pandas as pd + +from hydrology.data import nwm + + +def test_compare_nwm_usgs_returns_plot_data(monkeypatch): + nwm_index = pd.date_range("2024-01-01", periods=4, freq="D", tz="UTC") + nwm_df = pd.DataFrame({"streamflow_cfs": [100.0, 120.0, 130.0, 140.0]}, index=nwm_index) + usgs_df = pd.DataFrame( + {"value": [95.0, 125.0, 128.0, 145.0]}, + index=pd.date_range("2024-01-01", periods=4, freq="D", tz="UTC"), + ) + + class FakeClient: + def get_analysis(self, site_id, start_date, end_date): + return nwm_df + + monkeypatch.setattr(nwm, "NWMClient", FakeClient) + monkeypatch.setattr(nwm, "fetch_daily_values", lambda *args, **kwargs: usgs_df) + + result = nwm.compare_nwm_usgs("12422500", "2024-01-01", "2024-01-04") + + assert result is not None + assert result.n_observations == 4 + assert result.nwm_data is not None + assert result.nwm_data["streamflow_cfs"].tolist() == [100.0, 120.0, 130.0, 140.0] diff --git a/tests/test_reach_gain_loss.py b/tests/test_reach_gain_loss.py new file mode 100644 index 0000000..c792e4e --- /dev/null +++ b/tests/test_reach_gain_loss.py @@ -0,0 +1,22 @@ +import pandas as pd + +from hydrology.analysis.reach_gain_loss import classify_reach_gain_loss, summarize_reach_gain_loss + + +def test_summarize_reach_gain_loss_classifies_gaining_reach(): + index = pd.date_range("2021-08-01", periods=10, freq="D") + upstream = pd.Series([100] * 10, index=index) + downstream = pd.Series([130] * 10, index=index) + + result = summarize_reach_gain_loss(upstream, downstream, reach_km=10, drainage_area_sqmi=100) + + assert result["median_gain_cfs"] == 30 + assert result["median_gain_cfs_per_km"] == 3 + assert result["classification"] == "gaining" + assert result["confidence"] == "high" + + +def test_classify_reach_gain_loss_uses_deadband(): + assert classify_reach_gain_loss(0.2, deadband_cfs=1.0) == "neutral" + assert classify_reach_gain_loss(5.0, deadband_cfs=1.0) == "gaining" + assert classify_reach_gain_loss(-5.0, deadband_cfs=1.0) == "losing" diff --git a/tests/test_reach_topology.py b/tests/test_reach_topology.py new file mode 100644 index 0000000..49f9310 --- /dev/null +++ b/tests/test_reach_topology.py @@ -0,0 +1,62 @@ +from hydrology.analysis.reach_topology import ( + ReachPair, + build_reach_chain, + classify_pair_direction, + derive_adjacent_reaches, + validate_reach_pair, +) + + +def test_classify_pair_direction_accepts_downstream_metadata(): + sites = [ + {"site_id": "up", "direction": "upstream", "distance_km": 8.0}, + {"site_id": "down", "direction": "downstream", "distance_km": 12.0}, + ] + + assert classify_pair_direction("up", "down", sites, origin_site_id="origin") == "ordered" + + +def test_validate_reach_pair_flags_same_station(): + pair = validate_reach_pair("12422500", "12422500", related_sites=[]) + + assert pair.status == "invalid" + assert "same station" in pair.notes[0] + + +def test_validate_reach_pair_flags_unverified_direction(): + pair = validate_reach_pair( + "12422500", + "12424000", + related_sites=[{"site_id": "12424000", "direction": "upstream"}], + ) + + assert isinstance(pair, ReachPair) + assert pair.status == "unverified" + + +def test_build_reach_chain_orders_sites_from_upstream_to_downstream(): + selected = ["A", "B", "C"] + navigation_sites = [ + {"site_id": "C", "direction": "downstream", "distance_km": 20.0}, + {"site_id": "A", "direction": "upstream", "distance_km": 15.0}, + {"site_id": "B", "direction": "upstream", "distance_km": 5.0}, + ] + + chain = build_reach_chain(selected, navigation_sites, origin_site_id="origin") + + assert [station.site_id for station in chain.stations] == ["A", "B", "C"] + assert chain.status == "verified" + + +def test_derive_adjacent_reaches_returns_continuum_pairs(): + selected = ["A", "B", "C"] + navigation_sites = [ + {"site_id": "A", "direction": "upstream", "distance_km": 15.0}, + {"site_id": "B", "direction": "upstream", "distance_km": 5.0}, + {"site_id": "C", "direction": "downstream", "distance_km": 20.0}, + ] + chain = build_reach_chain(selected, navigation_sites, origin_site_id="origin") + + reaches = derive_adjacent_reaches(chain) + + assert [(reach.upstream_site_id, reach.downstream_site_id) for reach in reaches] == [("A", "B"), ("B", "C")] diff --git a/tests/test_signatures.py b/tests/test_signatures.py new file mode 100644 index 0000000..e446362 --- /dev/null +++ b/tests/test_signatures.py @@ -0,0 +1,24 @@ +import numpy as np +import pandas as pd + +from hydrology.analysis.signatures import compute_hydrologic_signatures + + +def test_compute_hydrologic_signatures_returns_core_metrics(): + index = pd.date_range("2020-01-01", periods=366, freq="D") + flow = pd.Series(100 + 30 * np.sin(np.linspace(0, 2 * np.pi, 366)), index=index) + + result = compute_hydrologic_signatures(flow) + + assert result["n_days"] == 366 + assert result["mean_flow"] > 0 + assert result["q05"] > result["q50"] > result["q95"] + assert 0 <= result["baseflow_index_lh"] <= 1 + assert result["richards_baker_flashiness"] >= 0 + assert 1 <= result["peak_month"] <= 12 + + +def test_compute_hydrologic_signatures_handles_empty_series(): + result = compute_hydrologic_signatures(pd.Series(dtype=float)) + + assert result == {} diff --git a/tests/test_stage_discharge.py b/tests/test_stage_discharge.py index ad43955..00f1794 100644 --- a/tests/test_stage_discharge.py +++ b/tests/test_stage_discharge.py @@ -12,6 +12,10 @@ from hydrology.analysis.stage_discharge import ( fit_powerlaw_rating_curve, fit_offset_powerlaw, + fit_best_rating_curve, + evaluate_rating_params, + predict_rating_discharge, + season_labels, flow_duration_curve, classify_flow_regime ) @@ -110,6 +114,61 @@ def test_offset_insufficient_data(self): assert np.isnan(A) +class TestFitBestRatingCurve: + """Tests for fit_best_rating_curve model selection.""" + + def test_prefers_offset_when_h0_matters(self): + """Sites like 14018500 need H0; simple power-law R² collapses.""" + np.random.seed(0) + H0 = 1.45 + H = np.linspace(1.6, 8.0, 80) + Q = 40.0 * (H - H0) ** 2.1 + Q = Q * (1 + np.random.normal(0, 0.03, size=H.size)) + idx = pd.date_range("2015-01-01", periods=len(H), freq="D") + stage = pd.Series(H, index=idx) + discharge = pd.Series(Q, index=idx) + + fit = fit_best_rating_curve(stage, discharge, min_points=20) + assert fit["model"] == "offset_powerlaw" + assert fit["R2"] > 0.95 + assert abs(fit["H0"] - H0) < 0.4 + + def test_season_labels(self): + idx = pd.DatetimeIndex(["2020-01-15", "2020-04-01", "2020-07-04", "2020-10-31"]) + seasons = season_labels(idx) + assert list(seasons.values) == ["DJF", "MAM", "JJA", "SON"] + + +class TestEvaluateRatingParams: + """Manual / interactive rating parameter scoring.""" + + def test_perfect_offset_scores_near_one(self): + H0 = 1.2 + H = np.linspace(1.5, 6.0, 60) + A, B = 25.0, 1.8 + Q = A * (H - H0) ** B + stage = pd.Series(H) + discharge = pd.Series(Q) + + scored = evaluate_rating_params(stage, discharge, A, B, H0) + assert scored["n_points"] == 60 + assert scored["R2"] > 0.999 + assert scored["rmse"] < 1e-6 + assert "H −" in scored["equation"] or "H -" in scored["equation"].replace("−", "-") + + def test_wrong_params_lower_r2(self): + H = np.linspace(1.0, 5.0, 40) + Q = 10.0 * H ** 2.0 + good = evaluate_rating_params(H, Q, 10.0, 2.0, 0.0) + bad = evaluate_rating_params(H, Q, 10.0, 0.5, 0.0) + assert good["R2"] > bad["R2"] + + def test_predict_rating_discharge_shape(self): + out = predict_rating_discharge(np.array([1.0, 2.0, 3.0]), A=2.0, B=1.5, H0=0.0) + assert out.shape == (3,) + assert np.all(out > 0) + + class TestFlowDurationCurve: """Tests for flow_duration_curve function.""" diff --git a/tests/test_temperature_context.py b/tests/test_temperature_context.py new file mode 100644 index 0000000..fc300b6 --- /dev/null +++ b/tests/test_temperature_context.py @@ -0,0 +1,25 @@ +from hydrology.analysis.temperature_context import classify_thermal_sensitivity + + +def test_classify_thermal_sensitivity_high_for_wide_unshaded_low_flow_reach(): + result = classify_thermal_sensitivity( + summer_flow_cfs=20, + channel_width_m=12, + canopy_cover_pct=15, + reach_gain_cfs=-3, + ) + + assert result["class"] == "high" + assert "low canopy cover" in result["drivers"] + assert "losing reach" in result["drivers"] + + +def test_classify_thermal_sensitivity_lower_for_shaded_gaining_reach(): + result = classify_thermal_sensitivity( + summer_flow_cfs=80, + channel_width_m=4, + canopy_cover_pct=85, + reach_gain_cfs=10, + ) + + assert result["class"] in {"low", "moderate"} diff --git a/tests/test_terminology.py b/tests/test_terminology.py new file mode 100644 index 0000000..04e6908 --- /dev/null +++ b/tests/test_terminology.py @@ -0,0 +1,22 @@ +from pathlib import Path +import subprocess + + +ROOT = Path(__file__).resolve().parents[1] + + +def test_tracked_text_uses_gage_terminology(): + forbidden = "gau" + "ge" + tracked = subprocess.check_output(["git", "ls-files"], cwd=ROOT, text=True).splitlines() + text_suffixes = {".py", ".md", ".txt", ".yml", ".yaml"} + + offenders = [] + for relative_path in tracked: + path = ROOT / relative_path + if path.suffix.lower() not in text_suffixes: + continue + text = path.read_text(encoding="utf-8", errors="ignore") + if forbidden in text.lower(): + offenders.append(relative_path) + + assert offenders == [] diff --git a/tests/test_validation_cases.py b/tests/test_validation_cases.py new file mode 100644 index 0000000..db207ff --- /dev/null +++ b/tests/test_validation_cases.py @@ -0,0 +1,14 @@ +from hydrology.analysis.validation import validate_range, validate_relative_error + + +def test_validate_relative_error_passes_within_tolerance(): + result = validate_relative_error("100yr flood", observed=443000, expected=475000, tolerance=0.10) + + assert result.status == "PASS" + assert abs(result.relative_error) < 0.10 + + +def test_validate_range_flags_out_of_range(): + result = validate_range("BFI", value=0.95, lower=0.55, upper=0.85) + + assert result.status == "FLAG"