cmrdesign

cmrdesign implements Conditional Minimax Regret (CMR) design rules in R and Python. The package is for applied researchers planning a main experimental wave after observing pilot data: pass pilot outcomes and assignment labels, get a recommended main-wave allocation and a worst-case regret certificate.

The methods accompany When and How to Pilot: Design Rules for Two-Wave Experiments by Juan C. Yamin.

This repository is software only. It does not contain paper replication code, raw research data, empirical calibration workflows, or paper-specific simulation output.

Installation

The R package is available on CRAN and the Python package is available on PyPI. For exact reproducibility, pin the version in your analysis environment.

R via CRAN:

install.packages("cmrdesign")

R development build via R-universe:

install.packages(
  "cmrdesign",
  repos = c("https://juancyamin.r-universe.dev", "https://cloud.r-project.org")
)

R development version from GitHub:

install.packages("remotes")
remotes::install_github("juancyamin/cmrdesign", subdir = "r")

Python via PyPI:

python -m pip install cmrdesign

For exact reproducibility:

python -m pip install cmrdesign==0.1.0

Python development version from GitHub:

python -m pip install "cmrdesign @ git+https://github.com/juancyamin/cmrdesign.git#subdirectory=python"

Quick Start

In a two-arm design, y is one vector of pilot outcomes and d marks the arm for each observation. Here the first 40 observations are treatment and the next 40 are control.

R:

library(cmrdesign)

set.seed(123)
d <- c(rep(1, 40), rep(0, 40))
y <- c(rbeta(40, 2, 5), rbeta(40, 4, 4))

fit <- cmr_two_arm(y, d, alpha = 0.05, method = "auto")
fit$pi
fit$U_CMR
summary(fit)

allocation <- realize_allocation(fit, n_main = 1000)
allocation$counts
allocation$realized_U_CMR

Python:

import numpy as np
import cmrdesign as cmr

rng = np.random.default_rng(123)
d = np.r_[np.ones(40), np.zeros(40)]
y = np.r_[rng.beta(2, 5, 40), rng.beta(4, 4, 40)]

fit = cmr.cmr_two_arm(y, d, alpha=0.05, method="auto")
fit.pi
fit.U_CMR
print(fit)

allocation = cmr.realize_allocation(fit, n_main=1000)
allocation.counts
allocation.realized_U_CMR

pi is the recommended treatment share for the main wave. U_CMR is the certificate: the worst-case regret of that allocation over the estimated confidence set for arm variances.

Which Function Should I Use?

If your design is...	Use...	Main inputs
One treatment and one control	cmr_two_arm()	y, d
Two arms with raw unbounded outcomes	cmr_unbounded()	y, d, psi
Several treatments sharing one control	cmr_multiarm()	y, arm, control_arm
Known strata with possibly different variances	cmr_stratified()	y, d, strata, strata_share
Multiple outcomes per unit	cmr_multiple_outcomes()	outcome matrix y, d, weights
Proxy or delayed primary outcome	cmr_proxy()	proxy_y, d, bridge constant zeta
Pilot versus main-wave sample-size planning	cmr_plan()	total n, pilot SD guesses
Integer main-wave counts from CMR shares	realize_allocation()	a CMR result, n_main

The direct rectangle functions, such as cmr_two_arm_from_rectangle() and cmr_multiarm_from_rectangle(), are useful for auditing or teaching. Applied users will usually pass pilot data directly and let the package estimate the confidence rectangle.

Choosing a Confidence Method

Method	Use when...	Notes
method = "auto"	You want the default applied behavior	Uses exact Bernoulli bounds for raw 0/1 outcomes and bounded-outcome bounds otherwise.
method = "bounded" or "mp"	Outcomes are bounded, usually normalized to [0, 1]	Uses Maurer–Pontil variance bounds.
method = "bernoulli"	Outcomes are truly binary and coded 0/1	Uses exact folded-binomial variance bounds.
method = "mtr"	You specifically want Martinez-Taboada–Ramdas bounds	Uses the pilot row order, so do not sort outcomes before calling it.
method = "unbounded"	Two-arm outcomes are raw finite values rather than bounded-scale values	Requires a kurtosis bound psi; use cmr_unbounded() for the clearest API.

For non-unit bounded outcomes, use normalize = TRUE in R or normalize=True in Python with known lower and upper support bounds. If those bounds are omitted, the package falls back to the pilot minimum and/or maximum with a warning; that convenience normalization is exploratory and does not carry the finite-sample bounded-outcome CMR guarantee. For two-arm raw finite outcomes without known support, consider cmr_unbounded(..., psi = ...), which requires a bounded-kurtosis input psi and may be conservative or return no finite CMR certificate. If a binary outcome is coded as something other than 0/1, recode it to 0/1 or explicitly set method = "bernoulli".

Interpreting Results

Most CMR result objects contain:

• pi: recommended main-wave allocation. In two-arm designs this is the treatment share; in multi-arm or stratified designs it can be a vector or matrix of shares.
• U_CMR: the worst-case regret certificate over the confidence set.
• rectangle or confidence_set: the variance uncertainty set used by the rule.
• method: the confidence-rectangle method actually used after auto dispatch.
• diagnostics: solver and edge-case information, such as whether the confidence set collapsed or became a full no-information rectangle.

Use realize_allocation(fit, n_main = ...) in R or realize_allocation(fit, n_main=...) in Python to convert continuous CMR shares into integer main-wave counts. When possible, the helper recomputes U_CMR at the realized shares so the rounded design has its own audit value.

CMR is a design rule for allocating the next experimental wave. It is not a treatment-effect estimator, and U_CMR is not a treatment-effect confidence interval.

Examples and Docs

All examples use simulated data.

• Quick Start: shortest two-arm example.
• R package site: R reference pages and rendered vignettes.
• Choosing a method: auto, bounded, Bernoulli, MTR, and unbounded rules.
• Methods: implementation details and supported extensions.
• Pilot planning: pilot/main-wave sizing screens from Appendix E of the accompanying paper (Yamin 2026).
• FAQ: input conventions and common edge cases.
• R examples and Python examples: simulated examples for each supported design.
• R vignettes in r/vignettes: applied tutorials for the core two-arm rule, confidence-method variants, extensions, and pilot planning.

Feedback

cmrdesign is an early public release. Applied-user feedback is especially useful as the package evolves:

• Bug reports: incorrect results, installation failures, solver errors, or R/Python inconsistencies.
• Usage questions: help choosing between CMR functions, confidence methods, or input formats.
• General feedback: comments on names, return objects, examples, documentation, or applied workflow.

Please use simulated, public, or redacted data in GitHub issues.

Citation

If you use cmrdesign, please cite the paper and the software:

@misc{yamin2026pilot,
  title = {When and How to Pilot: Design Rules for Two-Wave Experiments},
  author = {Yamin, Juan C.},
  year = {2026},
  doi = {10.48550/arXiv.2607.16982},
  url = {https://arxiv.org/abs/2607.16982}
}

@manual{cmrdesign2026,
  title = {cmrdesign: Conditional Minimax Regret Design Rules},
  author = {Yamin, Juan C.},
  year = {2026},
  note = {R package version 0.1.0},
  url = {https://CRAN.R-project.org/package=cmrdesign}
}

For Contributors

cmrdesign is a public R and Python package. The R package is distributed on CRAN; the two APIs are intended to remain parallel, and cross-language JSON fixtures check that the implementations return the same numerical results on shared cases.

spec/        Shared math/API specs and cross-language fixtures.
validation/  Independent reference and provenance checks.
r/           R package.
python/      Python package.
examples/    Simulated examples in R and Python.
docs/        User-facing documentation.