Skip to content

Dev - #14

Merged
mrustl merged 41 commits into
mainfrom
dev
Aug 7, 2026
Merged

Dev#14
mrustl merged 41 commits into
mainfrom
dev

Conversation

@mrustl

@mrustl mrustl commented Jun 3, 2026

Copy link
Copy Markdown
Member

No description provided.

claude and others added 30 commits June 3, 2026 12:28
ET0 fix: both full workflows now divide the daily ET0 by its interval in hours
(period_et = 24) before writing //Kurven/ET0, mirroring the rain conversion.
The kernel reads the ET0 curve as a mm/h rate, so unconverted daily values were
integrated 24x too high.

RAM: run_one() now thins each run to its single optimisation row immediately
(get_simulation_results_optim(lean = TRUE) + add_overflow_events_and_waterbalance)
and returns it; run_scenarios() collects the one-row tibbles and the analyse
chunk just binds them. This removes the get_simulation_results_optim_parallel()
pass that loaded every run's full time series into memory at once. New 'lean'
arg on get_simulation_results_optim() reads only the fields the optimisation
summary needs (element rates + both water balances), nulling states/meta/
connected-area rates; its intro message is gated behind debug.
Apply the same memory fix as Wien/BadAussee: run_one() now thins each run to
its optimisation row immediately (get_simulation_results_optim(lean = TRUE) +
add_overflow_events_and_waterbalance) and returns it; run_scenarios() collects
the rows and the analyse chunk binds them, replacing the
get_simulation_results_optim_parallel() pass that held every run's full time
series at once. No ET0 unit change here: the Eisenstadt workflows use the
base.h5 constant ET0 placeholder (0.2 mm/h) and the template rain (already
mm/h), so there is no mm/d daily series to convert.
Summary page linking the per-site brute-force outputs (workflow html, result
tables, CSVs, interactive plots). Added to .Rbuildignore so R CMD check does
not build/execute it; render it manually into the assembled results directory.
Result-file links match each workflow's paths$modelname (Wien, BadAussee,
Eisenstadt_2005).
Since the workflows now thin per run inside run_one(), reading happens inside
the future_lapply batch. A result file that exists but cannot be opened/read
(engine crashed mid-write for a scenario, or a transient lock) previously threw
H5File.open() 'unable to open file' and aborted the whole render at the
run_model chunk. Wrap the open+read in an inner function (own on.exit for handle
cleanup) plus tryCatch: such a file is now treated like a missing one -- warn,
name the scenario, return NULL -- so add_overflow_events_and_waterbalance()
emits an NA row and the batch completes.
R CMD check failed with a codoc WARNING because the .Rd still documented the
old signature (no 'lean'). Regenerate the usage block and add the \item{lean}
documentation to match R/get_simulation_results_optim.R (devtools::document()
equivalent), clearing the only WARNING (the 3 NOTEs are pre-existing).
to do: adapt Eisenstadt to the same parameter
Consistency:

- workflow_eisenstadt-2005.Rmd and workflow_eisenstadt-2005_neu.Rmd
  now unconditionally set
  //Massnahmenelemente/Mulde_Rigole/Parameter_Evapotranspiration/LAI_LeafAreaIndex
  = 3.9 (Hoernschemeyer grass value, Water 2023, 15, 2840, Tab. 6,
  plant type 5). Wien already sweeps LAI over c(3.9, 8.5); Bad
  Aussee uses the same sweep grid as Wien; Eisenstadt was the
  outlier still running on the base.h5 default 8.5.

New feature:

- plot_cost_vs_overflow_volume() -- new exported ggplot helper
  mirroring plot_wb_tradeoff_overflows() for cost-aware optimisation.
  x = cost_total (EUR), y = overflow volume in m3 (computed from
  sum_overflows [mm] * mulde_area [m2] / 1000), points coloured
  discretely by n_overflows using the same 0..x / ">x" palette and
  top legend. Plotly tooltip carries the cost breakdown
  (cost_excavation / _profiling / _filter / _storage / _total) plus
  the varying param_grid entries.

- Each of the four case-study workflow vignettes
  (workflow_wien, workflow_badaussee, workflow_eisenstadt-2005,
  workflow_eisenstadt-2005_neu) now renders the plot as
  simulation_results_optimisation_<site>_cost-vs-overflow-volume.html
  right after the existing water-balance render, matching its PDF /
  saveWidget pattern exactly.

- vignettes/index.Rmd gains a new "Kosten vs. Überlaufvolumen"
  section under "Interaktive Visualisierungen" that links to the
  three top-level sites (Eisenstadt 2005, Wien, Bad Aussee).

https://claude.ai/code/session_014QrjF51tg7cMVmsgjmfPNG
Builds on the cost-vs-overflow-volume plot (PR #15) with a new
cost-by-overflow-count boxplot (three best-selection variants), richer
tooltips and i18n parameter labels, and fixes the Eisenstadt cost
pipeline the PR review flagged.

Bug fix (PR #15 review blocker):

- vignettes/workflow_eisenstadt-2005.Rmd now pipes the joined
  optimisation results through kwb.raindrop::compute_costs(), like the
  Wien and Bad Aussee workflows already did. Without it the vignette's
  plot_cost_vs_overflow_volume() call aborted with "missing column(s):
  cost_excavation, ..." on Windows, so the cost PDF/HTML was never
  produced and the exported CSV lacked the cost columns.

New plot -- plot_cost_overflow_boxplot() (exported):

- Boxplot of total construction cost (EUR, y) per number of overflow
  events (x). Counts 0..x each get their own box (x = max_n_overflows,
  as in the sibling plots); higher counts collapse into a single ">x"
  catch-all box (furthest right, red), keeping the axis readable for the
  long-tailed 15-year runs (Wien / Bad Aussee reach several hundred
  overflow events). The ">x" box highlights the scenario with the fewest
  overflow events above x (closest to valid), best_by breaking ties.
- Individual scenarios are overlaid as jittered points whose size
  scales with a chosen variable (size_by): the overflow volume (m3,
  default) or the element evapotranspiration share (%). The size scale
  is calibrated to the valid region (0..x) and capped, with a minimum
  size, so the many-overflow outliers of the ">x" box do not shrink the
  valid-region points to invisible dots.
- One best scenario per box is highlighted with a black-outlined diamond
  in that box's group colour (so its tooltip inherits the group colour)
  and the best of all boxes are joined by a frontier line (mark_best /
  connect_best). best_by picks the objective, cost as tie-breaker --
  "min_cost" (cheapest), "min_overflow" (smallest overflow volume) or
  "max_evapotranspiration" (highest evapotranspiration) -- so the three
  variants trace three different frontier lines; label_best annotates
  the marker ("NN m3 / NN %" or "NN %").
- Each case-study vignette loops the three variants into
  *_cost-by-overflows-boxplot-{cheapest,min-overflow,max-evap}.html,
  all linked from vignettes/index.Rmd, which now groups the cost plots
  under one "Kosten" heading with sub-points (toc_depth 4).

Tooltip enrichment (shared helpers in R/cost_tooltip.R):

- The plotly tooltip of plot_cost_vs_overflow_volume() now carries the
  element water balance (evapotranspiration / infiltration / overflow,
  all in %) in addition to the cost breakdown, and names the chosen
  storage type on its own bold line, bilingually ("Sickerbox /
  Infiltration box" or "Schotterrigol / Gravel trench").
- The "varying parameters" block is translated via the new exported
  default_param_labels() helper -- a hovered point shows e.g.
  "Muldenflaeche [m2]=125" (de) / "Swale area [m2]=125" (en) instead of
  the raw "mulde_area=125"; override with param_labels =.
- cost_tooltip_labels() / cost_tooltip_text() / build_varying_param_html()
  are shared so both cost plots emit byte-identical tooltips.

Code hygiene:

- All non-ASCII characters in R code (string literals in the plot
  functions and in the vignette code chunks) are unicode-escaped
  (\uXXXX); the few non-ASCII code comments were rewritten in plain
  ASCII ("Hoernschemeyer", "2xN"). Markdown prose keeps UTF-8.

New exports: plot_cost_overflow_boxplot(), default_param_labels().
Both plot_cost_vs_overflow_volume() and plot_cost_overflow_boxplot()
now append the share of scenarios meeting the validity criterion
(n_overflows <= x) to the plot title, e.g.
"Kosten vs. Ueberlaufvolumen (39 % mit <= 5 Ueberlaeufen)" /
"Cost vs. overflow volume (39 % with <= 5 overflows)".
It goes in the title, not a ggplot subtitle: ggplotly drops subtitles,
so the share would be lost in the interactive HTML the vignettes export.
The scatter title drops its old "(Anzahl Ueberlaeufe <= x)"
parenthetical, which the share now supersedes.
vignettes/index.Rmd now spells out the per-site validity threshold that
drives this share: Eisenstadt <= 1 (1-year simulation), Wien / Bad
Aussee <= 5 (15-year rain/ET series).
…flow-plot

Cost-vs-overflow-volume plot + Eisenstadt LAI 3.9 consistency
New optimise_swale_design_simultaneous(): penalised Nelder-Mead search
(stats::optim, no new dependency) over mulde_area, mulde_height and
storage_height at once, instead of per-parameter bisection. Infeasible
designs are graded-penalised rather than excluded, so the simplex can
trade parameters against each other and the method does not rely on the
per-parameter monotonicity assumption.

Engine runs stay bounded via tolerance snapping (shared evaluation
cache across all x targets), a deterministic multistart (prior warm
start, previous-target optimum, one anchor start per storage level,
space-filling points; equal budget slices per start) and a final
lattice polish. Same interface and result schema as
optimise_swale_design(); a pairwise dominance check replaces the
bisection's volume referee.

- tests mirroring the bisection suite (brute-force reference, implicit
  storage escalation, infeasibility, warm start, max_total_depth) plus
  a mutual cross-check between both optimisers
- workflow_optimisation vignette: new section running the simultaneous
  search for all sites and tabulating cost deltas vs. bisection
- NEWS entry, docs and cross-references

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
Separate the two optimisation workflows: workflow_optimisation.Rmd is
restored to bisection-only (plus a pointer paragraph), the simultaneous
search moves into the new vignette workflow_optimisation_simultaneous.Rmd
(Nelder-Mead sweep for all sites, cell-by-cell comparison against the
bisection CSV export when present, and a method benchmark at x = 1).

optimise_swale_design_simultaneous() gains a method argument with two
alternative optimisers for comparison purposes:
- diff_evolution: compact DE/rand/1/bin, deterministic via an internal
  Park-Miller generator (seed argument); R's global RNG stays untouched
- halton_search: quasi-random space-filling baseline (Halton sequence)

All methods share the penalised objective, the evaluation cache, the
tolerance snapping and a new multi-valley lattice polish (accelerated
8/4/2/1-tolerance pattern descent from the cheapest feasible design of
every storage level visited - the storage axis separates cost valleys
that single coordinate steps cannot cross). Results carry a method
column.

Tests: per-method brute-force reference comparison, DE determinism and
.Random.seed invariance, start-configuration agreement within search
tolerances; 137 tests green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
Confirmed findings from a multi-lens review (algorithms, R semantics,
doc consistency, test quality), all reproduced before fixing:

- both optimisers crashed in the final arrange() when every cell was
  analytically infeasible (e.g. a strict max_total_depth): the empty
  evaluation cache produced a 0-column tibble; now they return the
  documented infeasible_within_bounds rows with a 0-row evaluations
  attribute
- a max_total_depth that compresses the gravel-trench bounds to a
  single admissible storage height (gb[2] == gb[1]) was misreported as
  infeasible without any evaluation; the degenerate axis is now solved
  (encode() guards the 0/0 division) in both optimisers
- prior_start_design(): NA rows in prior results injected all-NA rows
  via NA logical indexing and crashed compute_costs(); all filter
  conditions are NA-guarded now, and an all-NA-cost selection returns
  NULL instead of a 0-row frame that broke encode()
- DE mutation indices r1/r2/r3 could coincide (P ~ 1/11 for a zero
  difference vector); now pairwise distinct as DE/rand/1/bin requires
- NEWS: the simultaneous-vignette bullet had split the optimiser
  sub-bullet list, re-parenting make_swale_runner()/stack_levels()
  under the wrong bullet; docs now also state the 6-seed cap of the
  multi-valley polish instead of claiming "every storage level"

New tests: empty search space (both optimisers), degenerate gravel
axis, NA-containing prior, max_evals budget effect, seed-effect canary
on the search path, result attributes, single-type storage_spec for
all three methods. 160 tests green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
The optimisation vignettes died on the user's machine with HDF5
errno = 28 ("No space left on device"): every engine run creates its
own scenario directory (a copy of base.h5 as input plus all output
HDF5s) under tempdir() and nothing was ever removed. The bisection's
~15 runs per cell masked this; the simultaneous search (60-120 runs
per cell, ~124 scenarios per task across 6 parallel tasks with the
15-year Wien/Bad Aussee models) fills the temp drive mid-sweep.

The runner only ever uses the thinned one-row result, so it now
deletes the scenario's input copy and output directory right after
that row has been read (new argument cleanup = TRUE; failed runs keep
their files for debugging). Both optimisation vignettes document the
disk behaviour and the recovery step after an aborted run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
A full Nelder-Mead sweep is ~400-700 engine runs per task (2-3 h per
task at 15 s/run for the Wien / Bad Aussee models, more wall time when
fewer cores than tasks are free) and ran completely silent inside the
future workers - indistinguishable from a hang. Both compute chunks
now show a progressr bar ticking once per engine run across the worker
boundary (per-task quota topped up on completion so the bar ends at
exactly 100%). The runtime paragraph states the real expectation
(2-5 h), the site list gains a commented quick-test switch (Eisenstadt
only, minutes instead of hours) and max_evals is exposed in the
search-space chunk as the runtime lever.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
…trip per run

An Eisenstadt-only quick test still took close to an hour: the ~2 s
engine time per run was dwarfed by per-run overhead - every single run
copied base.h5, read ALL datasets via h5_read_values(), rewrote ALL of
them via h5_write_values() (for Wien / Bad Aussee including the
15-year rain series), and spawned a new process, with Windows virus
scanning on top of each new file.

The runner now prepares a site master file once on first call
(base.h5 + calculation settings + ET/rain time series + Growth/Shading
end-time fix) and each run copies that master and writes only its ~15
small parameter datasets (geometry, storage soil preset, kf/Psi, LAI,
result path; the base.h5 rain curve is cached once and rescaled only
when rain_factor != 1). Written file contents are identical to before;
untouched datasets now stay bit-identical to base.h5 instead of going
through a read/write round trip. rain_factor stays ignored when own
rain series are provided, as documented.

The vignette's quick-test note states realistic timing and points to
max_evals and a virus-scanner exclusion for the temp folder as the
remaining levers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
Comparing the Eisenstadt quick-test results against the bisection
showed the simultaneous NM optimiser landing up to 8.8% above the
bisection optimum in three cells (box x=0, trench x=0, trench x=5).
Two structural gaps in the polish, not search-budget issues:

- it could only step each axis straight down, so it could not slide
  ALONG the feasibility boundary (trade the expensive lever, area,
  down against the cheap one, mulde_height, up) - box x=0 got stuck at
  (61 m2, depth 200) where (55 m2, depth 300) is cheaper
- single steps could not cross a +1 counting-wobble band even when the
  whole lower mulde_height range is feasible again below it (trench
  x=5: stuck at depth 270 while depth 100 is feasible and ~800 EUR
  cheaper)

Each polish round now additionally proposes a boundary slide
(area - step, mulde_height at its maximum) and a mulde_height floor
probe (area, height_bounds[1]) - both plain evaluated candidates, so
no monotonicity assumption enters the simultaneous optimiser. On the
synthetic brute-force benchmark all three methods now converge to the
same lattice optimum (worst ratio 1.012, previously NM 1.016 /
DE 1.028 / Halton 1.081) at ~12% more runs for NM, and a synthetic
wobble-band scenario lands within 0.9% of the reference. 160 tests
green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
The bisection makes two distinct assumptions, and only one was
documented: besides hydraulic monotonicity (needed for interval
halving, immune to cost changes) its search ORDER encodes a cost
hierarchy - shrink area first at maximum depth, escalate storage only
when area is pinned at its bound, shrink depth last. That order picks
the corner of the feasibility boundary that is optimal for the default
rates (per mm of capacity and m2: depth ~0.07 EUR, box storage
~0.44 EUR, area pays all four cost components at once); cost_rates
only prices the found designs afterwards, it does not steer the
search. With strongly different rates the optimal corner moves to
places the order never visits: with box material at 5 EUR/m3 the
bisection returns 20832 EUR (155 m2, storage 300) where the
simultaneous search - which carries cost_rates inside its objective -
finds 12682 EUR (63 m2, storage 1200), 39% cheaper. Added as a
regression test.

Docs now state this in optimise_swale_design(), the bisection
vignette's cost-rates section, and the simultaneous vignette's
Einordnung, which also distinguishes the two causes of "simultaneous
systematically cheaper": violated monotonicity (model alarm) vs. a
cost hierarchy that no longer matches the chosen rates (use the
simultaneous optimiser as the primary method for cost sensitivity
studies). 164 tests green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
Follow-up to documenting the hard-coded cost hierarchy: for the current
three levers the order does not need to be hard-coded at all. With the
capacity model V ~ area * (mulde_height + porosity * storage_height)
and every cost component proportional to area, each lever has a
marginal cost per mm of capacity, the area cancels out of the
comparison, and the cost-optimal corner of the feasibility boundary is
computable analytically from the rates:

- maximising mulde_height first is optimal for ANY rates under this
  cost model (it costs only excavation while area pays all four
  components plus the capacity-free filter depth) - so that part of
  the order is now provably rates-independent, not assumed
- the starting storage level is the rate-dependent choice: solve_cell
  picks the level with minimal specific cost (EUR per mm capacity)
  instead of always the smallest; the continuous trench compares its
  endpoints (linear-fractional, endpoint-optimal). Under default rates
  the proxy picks the smallest level, reproducing the previous
  behaviour exactly.

default_storage_spec() now carries the layer porosity (box 0.95,
trench 0.3) that the proxy needs; specs without it keep the legacy
order. With box material at 5 EUR/m3 the bisection now finds the
63 m2 / storage 1200 corner for 12.7k EUR in 15 runs - previously
20.8k EUR (155 m2 / storage 300), and on par with the simultaneous
optimiser's 12.7k EUR at ~130 runs. The regression test now asserts
both optimisers agree under changed rates, the legacy fallback, and
unchanged default-rate behaviour. The proxy is first-order and only
ranks capacity-additive levers - parameters with nonlinear hydraulic
effects (e.g. variable filter kf) remain the domain of the
simultaneous optimiser; docs and both vignettes updated accordingly.
167 tests green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
The final mulde_height trim has two distinct sources of slack, and
only one of them is real: at low overflow targets it merely harvests
the area search's tolerance overshoot (a search artefact worth ~18%
of what finer area resolution would recover, since height only saves
excavation while area pays all four cost components), at high targets
it exploits the wide overflow-count plateaus where height has almost
no hydraulic effect (real savings, e.g. 300 -> 100 mm at Eisenstadt
x = 5, ~600 EUR, unreachable via finer area search because the next
area step down immediately overshoots the target).

Document the cheap fix for the artefact part: thanks to the
bisection, halving area_tol costs exactly one additional engine run
per area search while halving the worst-case cost overshoot
(~area_tol x specific cost per m2, a few percent at the default
2 m2). Noted in the area_tol roxygen docs and as a comment on the
vignette's search-space knob.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
- DESCRIPTION: version 0.0.0.9000 -> 0.1.0
- NEWS.md: development-version header becomes the 0.1.0 release header
- CITATION.cff: citation metadata for GitHub's "Cite this repository"
  and the Zenodo GitHub integration (author with ORCID, MIT, keywords,
  abstract noting that the proprietary Tandler engine is downloaded
  from the separate kwb.raindrop.binaries repository and is not part
  of this package)
- .zenodo.json: explicit Zenodo deposit metadata (creators, license,
  keywords, funding note for RAINDROP / GZ C300428)
- inst/CITATION: R-style citation() entry
- .Rbuildignore: exclude the repo-level citation files from the built
  package (inst/CITATION ships with it)

The GitHub release v0.1.0 itself is cut from the default branch after
merging; enable the Zenodo webhook for KWB-R/kwb.raindrop beforehand
so the release is archived and receives a DOI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
claude and others added 11 commits August 7, 2026 09:19
- NEWS.md: the make_swale_runner() entry now leads with an explicit
  behaviour note that cleanup = TRUE (the default) deletes each
  scenario's output HDF5s (Mulde_Rigole.h5, Dach.h5,
  Fehlerprotokoll.h5, ...) right after the thinned result row is read,
  and that cleanup = FALSE keeps them - so consumers are not surprised
  that only the returned tibble survives a run.
- test-optimise_swale_design_simultaneous.R: float comparisons
  (cost_total, mulde_area, storage_height, mulde_height) use
  expect_equal(tolerance = 0) instead of expect_identical - exact
  values without bitwise/type strictness; integer counts, strings and
  .Random.seed stay expect_identical.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
prepare_master() read //Kurven/Regen (and Growth_1/Shading_1)
unconditionally, so a base.h5 template without these datasets crashed
with a cryptic low-level error. Now the available datasets are listed
once and only existing curves are touched:

- timeseries_rain given: //Kurven/Regen must exist (clear error naming
  the template path otherwise); the Growth/Shading end-time fix is
  applied only to curves that exist and have the expected time/value
  shape
- no timeseries_rain: the template rain curve is cached only if
  present; a per-run rain_factor != 1 without a usable template curve
  now throws a clear error instead of silently not scaling

Requirements documented in the roxygen docs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017npvdq8XW1Lg1t2dNX9sGH
for simulataneous optimisation testing
vignettes/index.Rmd (brute-force link hub) and
vignettes/monotonicity_analysis.Rmd are deleted (they remain in the git
history): their relative links only work in the deploy structure on
raindrop.kompetenz-wasser.io and their result files never exist on
GitHub Actions, so the pkgdown-built copies were link-dead shells. The
complete rendered pages live on that server.

- New "Ergebnisse" navbar menu linking the published results (brute
  force, monotonicity analysis, optimiser vs. brute force).
- The two optimisation vignettes and the roxygen docs of
  find_min_feasible(), optimise_swale_design() and
  optimise_swale_design_simultaneous() now link the monotonicity
  analysis on that server instead of the removed vignette.
- Drop the stale vignettes/index.Rmd entry from .Rbuildignore and the
  mono_* mention from .gitignore.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017E2RpwmBRnnDHRmxjXRh4x
…d8ab

Remove deploy-only articles from pkgdown site, link results externally
The pkgdown navbar chrome is English (Get started, Reference, Articles,
Changelog), so the new menu is now "More results" with English item
labels (brute force, monotonicity analysis, optimiser vs. brute force).
The German link texts inside the German-language vignettes are kept.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017E2RpwmBRnnDHRmxjXRh4x
…d8ab

Use English labels for the external-results navbar menu
@mrustl
mrustl merged commit d67ef33 into main Aug 7, 2026
10 checks passed
Comment thread R/make_swale_runner.R
Comment on lines +101 to +105
prepare_master <- function(paths) {
mp <- file.path(paths$dir_input,
sprintf("%s_master.h5", scenario_prefix))
fs::dir_create(paths$dir_input, recurse = TRUE)
fs::file_copy(path = paths$path_base, new_path = mp, overwrite = TRUE)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential race condition on the shared master file when parallel workers target the same site.

The master path is derived only from paths$dir_input and scenario_prefix, so both future_lapply workers of the same site (different storage_type) point to the same <site>/input/o_master.h5. First call in each worker (there are two, one per storage type) will:

  1. fs::file_copy(base.h5 → o_master.h5, overwrite = TRUE)
  2. hdf5r::H5File$new(mp, mode = "a") — opens the shared master in append mode

If both prepare_master calls interleave, worker B's file_copy(overwrite=TRUE) can stomp on the master file A currently has open in append mode, and/or the second H5File$new(..., mode="a") may fail due to HDF5 file locking. The vignettes in workflow_optimisation.Rmd fan out over expand.grid(site, type), so both workers of a site race here.

Suggested fixes (in order of decreasing intrusiveness):

  • Include a per-worker discriminator in the master filename (e.g. sprintf("%s_master_%s.h5", scenario_prefix, Sys.getpid())), so each worker has its own; or
  • Serialise creation via a lockfile (fs::file_touch + while(...) polling) — brittle;
  • Document that make_swale_runner() is not safe to reuse a dir_input across concurrent workers.

The one-master-per-worker approach doubles disk use temporarily but eliminates the race entirely.

Comment thread R/read_site_timeseries.R
Comment on lines +40 to +41
timeseries_et$time[nrow(timeseries_et)] <-
ceiling(timeseries_et$time[nrow(timeseries_et)])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead ceiling() on an already-integer column.

timeseries_et$time was computed above (line 34–36) as as.integer(difftime(..., units = "hours")), which is already an integer. ceiling() on integer values is a no-op — this line does nothing (compare with line 53–54 for rain, where time is a double and ceiling actually rounds up).

Two options:

  • If the intent was to round up after as.integer's truncation, replace the earlier as.integer(...) with as.double(...) so the truncation doesn't happen first, then ceiling here matters.
  • If the intent is a no-op (ET is daily and always integer hours by construction, so nothing to round), drop lines 40–41 entirely.

Comment on lines +263 to +293
evap_all <- simulation_results_optimisation[["element.WB_Evapotranspiration_"]]
valid_mask <- !is.na(simulation_results_optimisation$n_overflows) &
simulation_results_optimisation$n_overflows <= x_int & !is.na(evap_all)
ref_idx <- if (any(valid_mask)) which(valid_mask) else seq_along(evap_all)
evap_min <- suppressWarnings(min(evap_all[ref_idx], na.rm = TRUE))
evap_min_scenario <- simulation_results_optimisation$scenario_name[
ref_idx[which.min(evap_all[ref_idx])]]

# Share of scenarios meeting the validity criterion (n_overflows <= x),
# appended to the auto-generated title (a plotly-safe place -- ggplotly
# drops ggplot subtitles).
valid_pct <- round(100 * mean(
simulation_results_optimisation$n_overflows <= x_int, na.rm = TRUE))
share_txt <- switch(lang,
de = paste0(valid_pct, " % mit <= ", x_int, " \u00dcberl\u00e4ufen"),
en = paste0(valid_pct, " % with <= ", x_int, " overflows"))
if (is.null(title)) {
title <- paste0(def_title, " (", share_txt, ")")
if (y_var == "cost_per_evap_pct") {
# name the reference of the marginal measure in the title
title <- paste0(title, "\n", switch(lang,
de = paste0("Referenz: minimale Evapotranspiration der g\u00fcltigen ",
"Szenarien (<= ", x_int, " \u00dcberl\u00e4ufe): ",
round(evap_min, 1), " % (Szenario ",
evap_min_scenario, ")"),
en = paste0("Reference: minimum evapotranspiration of the valid ",
"scenarios (<= ", x_int, " overflows): ",
round(evap_min, 1), " % (scenario ",
evap_min_scenario, ")")))
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

evap_min_scenario becomes character(0) when every evap_all entry is NA, and the title collapses to character(0).

When no scenarios are valid AND every value in evap_all is NA (pathological but possible for a fully-failed batch), the fallback ref_idx <- seq_along(evap_all) fires, then which.min(evap_all[ref_idx]) returns integer(0) (min on all-NA), so ref_idx[integer(0)] = integer(0) and evap_min_scenario = character(0).

At line 283–291 the German/English branch does paste0(..., evap_min_scenario, ...). paste0 with a character(0) argument returns character(0), which then flows into title <- paste0(title, "\n", character(0))character(0). Passing a character(0) title to ggplot2::labs() produces an unhelpful downstream error rather than the intended graceful degradation.

Suggested guard (minimal patch, right at the definition):

Suggested change
evap_all <- simulation_results_optimisation[["element.WB_Evapotranspiration_"]]
valid_mask <- !is.na(simulation_results_optimisation$n_overflows) &
simulation_results_optimisation$n_overflows <= x_int & !is.na(evap_all)
ref_idx <- if (any(valid_mask)) which(valid_mask) else seq_along(evap_all)
evap_min <- suppressWarnings(min(evap_all[ref_idx], na.rm = TRUE))
evap_min_scenario <- simulation_results_optimisation$scenario_name[
ref_idx[which.min(evap_all[ref_idx])]]
# Share of scenarios meeting the validity criterion (n_overflows <= x),
# appended to the auto-generated title (a plotly-safe place -- ggplotly
# drops ggplot subtitles).
valid_pct <- round(100 * mean(
simulation_results_optimisation$n_overflows <= x_int, na.rm = TRUE))
share_txt <- switch(lang,
de = paste0(valid_pct, " % mit <= ", x_int, " \u00dcberl\u00e4ufen"),
en = paste0(valid_pct, " % with <= ", x_int, " overflows"))
if (is.null(title)) {
title <- paste0(def_title, " (", share_txt, ")")
if (y_var == "cost_per_evap_pct") {
# name the reference of the marginal measure in the title
title <- paste0(title, "\n", switch(lang,
de = paste0("Referenz: minimale Evapotranspiration der g\u00fcltigen ",
"Szenarien (<= ", x_int, " \u00dcberl\u00e4ufe): ",
round(evap_min, 1), " % (Szenario ",
evap_min_scenario, ")"),
en = paste0("Reference: minimum evapotranspiration of the valid ",
"scenarios (<= ", x_int, " overflows): ",
round(evap_min, 1), " % (scenario ",
evap_min_scenario, ")")))
}
}
evap_all <- simulation_results_optimisation[["element.WB_Evapotranspiration_"]]
valid_mask <- !is.na(simulation_results_optimisation$n_overflows) &
simulation_results_optimisation$n_overflows <= x_int & !is.na(evap_all)
ref_idx <- if (any(valid_mask)) which(valid_mask) else seq_along(evap_all)
evap_min <- suppressWarnings(min(evap_all[ref_idx], na.rm = TRUE))
wm <- which.min(evap_all[ref_idx])
evap_min_scenario <- if (length(wm) == 0L) {
NA_character_
} else {
simulation_results_optimisation$scenario_name[ref_idx[wm]]
}
# Share of scenarios meeting the validity criterion (n_overflows <= x),
# appended to the auto-generated title (a plotly-safe place -- ggplotly
# drops ggplot subtitles).
valid_pct <- round(100 * mean(
simulation_results_optimisation$n_overflows <= x_int, na.rm = TRUE))
share_txt <- switch(lang,
de = paste0(valid_pct, " % mit <= ", x_int, " Überläufen"),
en = paste0(valid_pct, " % with <= ", x_int, " overflows"))
if (is.null(title)) {
title <- paste0(def_title, " (", share_txt, ")")
if (y_var == "cost_per_evap_pct") {
# name the reference of the marginal measure in the title
title <- paste0(title, "\n", switch(lang,
de = paste0("Referenz: minimale Evapotranspiration der gültigen ",
"Szenarien (<= ", x_int, " Überläufe): ",
round(evap_min, 1), " % (Szenario ",
evap_min_scenario, ")"),
en = paste0("Reference: minimum evapotranspiration of the valid ",
"scenarios (<= ", x_int, " overflows): ",
round(evap_min, 1), " % (scenario ",
evap_min_scenario, ")")))
}
}

Comment thread R/make_swale_runner.R
Comment on lines +135 to +142
for (curve in c("//Kurven/Growth_1", "//Kurven/Shading_1")) {
if (!curve %in% existing) next
cv <- h5_read_values(h5m, paths = curve)[[curve]]
if (is.data.frame(cv) && length(cv$time) >= 2) {
cv$time[2] <- max(timeseries_rain$time)
static_vals[[curve]] <- cv
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only cv$time[2] is updated — silently corrupts any Growth/Shading curve with more than 2 points.

The extension logic assumes the shipped Growth_1 / Shading_1 curves are exactly two-point ramps (start → end). For any curve with 3+ knots, points 3..n retain their original time values while point 2 is jumped to the rain end, producing a non-monotone (or otherwise physically wrong) curve that the engine reads without complaint.

Given the site templates the vignettes exercise are all 2-point ramps this hasn't bitten yet, but the length(cv$time) >= 2 guard reads as if any length were handled. Either:

  • tighten the guard to length(cv$time) == 2 and stop/skip otherwise, so the assumption is enforced; or
  • rescale all inner knots proportionally, e.g. cv$time <- cv$time * (max(timeseries_rain$time) / cv$time[length(cv$time)]).

At minimum, add a comment stating the 2-point assumption.

(length(starts) - si + 1))
stats::optim(starts[[si]], objective, method = "Nelder-Mead",
control = list(maxit = 200, reltol = 1e-4,
warn.1d.NelderMead = FALSE))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warn.1d.NelderMead = FALSE does nothing here — the search is 3-D.

That control is only consulted when par has length 1 (see ?stats::optim), so on a 3-parameter starting point it is silently ignored. Not a bug, but the flag reads as if it were suppressing something and future-maintainers will wonder what. Suggest removing it, or replacing with a brief comment noting that the 1-D branch is intentionally impossible here.

Suggested change
warn.1d.NelderMead = FALSE))
stats::optim(starts[[si]], objective, method = "Nelder-Mead",
control = list(maxit = 200, reltol = 1e-4))

Comment thread R/find_min_feasible.R
Comment on lines +128 to +149
if (!feasible(axis_hi)) {
n_hi <- eval_at(axis_hi)$n_overflows
anchor <- NA_real_
if (!is.na(n_hi) && n_hi <= x_max + wobble) {
offset <- axis_tol
repeat {
p <- axis_hi - offset
if (discrete) p <- ceiling(p)
if (p <= axis_lo) break
if (feasible(p)) {
anchor <- p
break
}
offset <- offset * 2
}
}
if (is.na(anchor)) {
status <- "infeasible"
} else {
axis_hi <- anchor
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wobble ladder only searches a geometric offset sequence (1, 2, 4, 8, …) below the upper edge — a strictly-monotone-but-decreasing-in-large-steps violation between the edge and lower bounds can be missed.

In discrete mode with axis_hi = length(grid) = 4 and wobble = 1, the ladder tests indices 3 then 2 (offset 1 then 2), then p = 0 <= axis_lo breaks. If only index 1 is feasible while 2 and 3 are not, the ladder declares infeasible even though a feasible level exists.

This is documented as a monotonicity assumption (the volume referee catches most violations) and the description at line 12–23 spells this out, so it's likely intentional. But the ladder's exponential search means the tolerance window shrinks the further below axis_hi the feasible anchor sits — worth calling out in the docstring so users know the wobble is only a neighbour fix, not a global rescue. Consider adding one sentence to the doc:

The ladder walks downward in offsets axis_tol, 2·axis_tol, 4·axis_tol, …; feasible anchors more than a few tolerance steps below the edge will only be found if strict monotonicity holds up to that step size.

Comment on lines +328 to +337
df <- df %>%
dplyr::mutate(
cost_per_evap_pct = dplyr::if_else(
.data[["element.WB_Evapotranspiration_"]] - evap_min > 0,
.data$cost_total /
(.data[["element.WB_Evapotranspiration_"]] - evap_min),
NA_real_
)
) %>%
dplyr::filter(!is.na(.data[[y_col]]))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cost_per_evap_pct uses .data$cost_total in the numerator regardless of the active y_var.

The definition is cost_total / (evap - evap_min), but when the caller is going to filter by y_col = "cost_per_evap_pct" anyway, this is fine. Just worth verifying: is cost_total the intended numerator, or should it be the raw y_col value? If y_col == "cost_per_evap_pct" gets fed back through this expression on a subsequent invocation, the divide-by-evap would compound.

In current usage the numerator is cost_total unconditionally, so this is not a bug today — flagging as a defensive-review concern only. Consider a short comment: # always cost_total (not y_col) — the *derivation* of the marginal cost measure.

} else {
0
}
u1 <- (area - area_bounds[1]) / diff(area_bounds)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

u1 = (area - area_bounds[1]) / diff(area_bounds) — silently NaN if degenerate area_bounds = c(v, v) are ever passed.

Unlike u2 (guarded at line 423 with up > height_bounds[1]) and u3 (guarded for discrete via index; guarded for continuous via gb[2] > gb[1]), there's no diff(area_bounds) > 0 guard on the area axis. Passing area_bounds = c(100, 100) would produce u1 = NaN, which then propagates through pmin/pmax (both return NaN) and into stats::optim() where it errors with an opaque message.

Not encountered in the shipped defaults, and the stopifnot() at line 255 would arguably be the right place to add diff(area_bounds) > 0 if you want to fail loudly rather than by silent NaN.

Comment on lines +44 to +72
test_that("Optimierer findet das Kostenminimum (Vergleich mit Brute-Force)", {
run <- synthetic_run_factory(demand = 3.6e5)
out <- optimise_swale_design(run, x_targets = 0:3,
fixed = test_fixed, verbose = FALSE)

expect_true(all(out$status == "ok"))
expect_false(any(out$monotonicity_warning))
# Zulaessigkeit: Ueberlaufziel eingehalten
expect_true(all(out$n_overflows <= out$x))

for (i in seq_len(nrow(out))) {
type <- out$storage_type[i]
stor <- if (type == "infiltration_box") c(300, 600, 900, 1200)
else seq(900, 3600, by = 25)
ref <- reference_optimum(run, type, out$x[i], stor)
expect_false(is.null(ref))
# innerhalb 5 % des (quasi-kontinuierlichen) Brute-Force-Optimums
expect_lte(out$cost_total[i], ref$cost_total * 1.05)
}

# Kosten-Wirksamkeits-Kurve: lockereres Ziel ist nie teurer
for (type in unique(out$storage_type)) {
cc <- out$cost_total[out$storage_type == type][order(out$x[out$storage_type == type])]
expect_true(all(diff(cc) <= 1e-9))
}

# Laufbudget: alle 8 Zellen zusammen deutlich unter Brute-Force-Groesse
expect_lte(attr(out, "n_runs_total"), 200)
})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice reference-optimum design — worth extending one edge that isn't currently covered.

The synthetic-model tests are impressively thorough. One case that isn't exercised in either test-optimise_swale_design.R or test-optimise_swale_design_simultaneous.R: the volume referee firing end-to-end (a monotonicity violation flowing all the way to monotonicity_warning = TRUE in the output tibble). test-find_min_feasible.R covers it for the primitive, but the plumbing that assigns res_a$monotonicity_violation and res_s$monotonicity_violation and OR's them into mono_warn in solve_cell() (R/optimise_swale_design.R:268, 287, 304) is not tested — a synthetic run_fn that returns a real non-monotonicity (n_overflows AND overflow volume increase with area) would catch a regression in that wiring.

Non-blocking; just a suggestion for a follow-up commit.

Comment on lines +120 to +124
kwb.utils::catAndRun(
messageText = sprintf("(%d/%d)) Reading results files for model run %s",
which(simulation_names == s_name),
length(simulation_names),
paths$dir_target_output),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which(simulation_names == s_name) inside the lapply returns multiple indices if simulation_names has duplicates, then sprintf errors out.

Purely defensive: shipping code and tests never pass duplicate scenario names, so this only matters for hand-written calls. A simple fix that stays byte-identical for the common (unique) case:

messageText = sprintf("(%d/%d)) Reading results files for model run %s",
                      match(s_name, simulation_names),
                      length(simulation_names),
                      paths$dir_target_output),

match() returns just the first index — always length 1.

Comment on lines +346 to +357
# Point size: calibrate the scale to the valid region (0..x) and cap the
# extreme ">x" values, otherwise the many-overflow outliers (overflow
# volumes of several thousand m3) shrink every valid-region point to an
# invisible dot. pmin() caps; scale_size() below adds a minimum size so even
# zero-volume points (the 0-overflow box) stay visible.
valid_size <- df[[size_col]][!is.na(df$n_overflows) & df$n_overflows <= x_int]
size_cap <- suppressWarnings(max(valid_size[is.finite(valid_size)]))
if (!is.finite(size_cap) || size_cap <= 0) {
size_cap <- suppressWarnings(max(df[[size_col]], na.rm = TRUE))
}
if (!is.finite(size_cap) || size_cap <= 0) size_cap <- 1
df$size_plot <- pmin(df[[size_col]], size_cap)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

size_cap semantics are only correct for size_by = "overflow_volume" — they invert the "larger = better" meaning for size_by = "evapotranspiration".

The clip-to-valid-region logic is justified for overflow volume: invalid scenarios have very large volumes that would compress the whole scale, so capping at the valid-region max keeps the valid points readable.

But for size_by = "evapotranspiration" (line 60–63 of the docstring: "larger points then mean more evapotranspiration, which is desirable"), invalid scenarios can have evapotranspiration higher or lower than valid ones — and if higher, capping them makes the desirable outliers look like the valid-region maximum, hiding the "outperformer that overflows too often" signal that the size scale is meant to convey.

Suggested guard: only apply the cap when size_by == "overflow_volume".

if (size_by == "overflow_volume") {
  valid_size <- df[[size_col]][!is.na(df$n_overflows) & df$n_overflows <= x_int]
  size_cap <- suppressWarnings(max(valid_size[is.finite(valid_size)]))
  if (!is.finite(size_cap) || size_cap <= 0) {
    size_cap <- suppressWarnings(max(df[[size_col]], na.rm = TRUE))
  }
  if (!is.finite(size_cap) || size_cap <= 0) size_cap <- 1
  df$size_plot <- pmin(df[[size_col]], size_cap)
} else {
  df$size_plot <- df[[size_col]]
}

#' @importFrom rlang .data
plot_cost_overflow_boxplot <- function(simulation_results_optimisation,
param_grid,
x = 5,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sibling default mismatch: x = 5 here vs. x = 1 in plot_cost_vs_overflow_volume / plot_cost_vs_evaporation.

The docstring (line 22–27) advertises the tooltip as identical to plot_cost_vs_overflow_volume, and both plots derive evap_min from "scenarios that satisfy n_overflows <= x", so the same tooltip line will actually differ between the two plots at their defaults. If a user renders both in a vignette without overriding x, they'll see the same scenario reported with different cost_per_evap numbers side by side.

Either align the defaults (both to x = 1 or both to x = 5), or add a note to both docstrings that the derived tooltip line depends on x and users should pass the same value to both.

@claude

claude Bot commented Aug 7, 2026

Copy link
Copy Markdown

Two more findings for this PR that I couldn't attach as inline comments because the target lines fall outside this PR's diff hunks (they're in unchanged prose / defaults of a heavily-changed file):

HIGH — R/plot_valid_design_space.R:164-172 — English title uses raw column names, no English axis-label dictionary

The English branch of txt$title (line 165-172) interpolates the raw x and y arguments (the column names), whereas the German branch above uses the resolved lab_x / lab_y. Combined with axis_labels_de (line 135-140) being German-only, plot_valid_design_space(..., lang = "en", x = "mulde_area", y = "mulde_height") produces:

"Valid solutions (Number of overflow events <= 5)\nin design space: mulde_area × mulde_height"

Suggested fix: route both languages through the shared default_param_labels(lang) dictionary and use lab_x / lab_y in the English paste0 too. Bonus: default_param_labels() covers filter_height, connected_area, rain_factor, lai, storage_type, bottom_hydraulicconductivity, all currently untranslated even in German.

HIGH — R/plot_wb_tradeoff_overflows.R:6-14, 38-42 — docstring off-by-one

The description says:

red for n_overflows >= x, shown as category ">=x"

and filter_n_gtx param doc:

If TRUE, scenarios with n_overflows >= x are removed before plotting.

But the code uses strict > (line 176 hi_lab <- paste0(">", x_int), line 182 n_overflows > x_int ~ hi_lab, line 175 n_overflows <= x_int), matching the sibling cost plots. A user setting x = 3 per the docs to drop >= 3 overflows actually keeps 0, 1, 2, 3. Since behaviour matches the siblings, the fix is docs-only: >=x>x, ">=x"">x", and the filter_n_gtx param description.


These are the last two of a batch of 14 review findings on this PR; the other 12 posted successfully as inline comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants