Conversation
MeasureBase 0.15 removed the origin machinery (`transport_origin`, `to_origin`, `from_origin`) and `NoTransport`, and now always provides `pwr_base`/`pwr_axes`/`pwr_size`, `NoFastInsupport` and `localmeasure`, so the `@static` compatibility shims can go. The pushforward origin accessors are replaced by direct field access on `BATPushFwdMeasure`, which is going away in the following commits anyway. Imports of the MeasureBase names the migration needs (`asmeasure`, `mbind`, `mintegrate_exp`, `productmeasure`, `restrict`, `logdensities`, `Half`, `StdExponential`, `StdLogistic`, `TransportFunction`) are added here. Created by generative AI.
MeasureBase 0.15 provides everything BAT's own measure layer used to duplicate, so the duplicates go away and BAT's measure types subtype `MeasureBase.AbstractMeasure` directly. Removed: the `BATMeasure` supertype, `BATDistMeasure` (use `asmeasure`), `BATPwrMeasure`, `BATPushFwdMeasure`, `BATWeightedMeasure`, `BATSuperpositionMeasure` (use `powermeasure`, `pushfwd`, `weightedmeasure`, `superpose`), BAT's `StandardUvUniform`, `StandardMvUniform`, `StandardUvNormal` and `StandardMvNormal` (use `StdUniform()`, `StdNormal()` and their powers), and the whole of `src/transforms/distribution_transform.jl` (783 lines, replaced by `transport_to`). `PosteriorMeasure`, `EvaluatedMeasure`, `DensitySampleMeasure` and `BispacedMeasure` stay, re-based on `AbstractMeasure`; a posterior is now a proper MeasureBase measure with `basemeasure` the prior and `logdensity_def` the likelihood. `batmeasure` becomes BAT's canonicalization: `asmeasure` for distributions, product measures for named tuples and `NamedTupleDist`s (with `ConstValueDist` entries becoming `Dirac`), and `PosteriorMeasure` for MeasureBase density measures. `distprod` builds such product measures, which changes what it returns (NEWS-worthy). The transform layer keeps `TransformIntent`, `TransformAlgorithm`, `bat_transform` and `transform_and_unshape`; only the bodies change: `UniformBased`/`NormalBased` are `transport_to(StdUniform()^n, μ)` resp. `transport_to(StdNormal()^n, μ)` with the power built explicitly, and transformed measures are MeasureBase pushforwards. ValueShapes stays BAT's sample storage layer. All ValueShapes support for MeasureBase measures (`varshape`, `unshaped`, shaping, `resultshape` of a transport, the `NamedTupleDist` conversion) is collected in the new `src/measures/measure_shapes.jl`, so that it can move into a ValueShapes extension of MeasureBase later. Two consequences of MeasureBase's out-of-support convention needed handling downstream: prior substitution loses the original prior's support, where transports produce infinite variates at which likelihoods need not be defined, so the substituted likelihood carries that support along (`BAT.SupportedDensity`); and `checked_logdensityof` now returns a zero density instead of throwing when a non-finite variate yields `NaN`, which algorithms exploring the variate space do produce. `DistributionsAD` and `ForwardDiffPullbacks` are no longer used. The test suite and the docs still refer to the removed names, they are adapted in a following commit. Created by generative AI.
`MeasureBase.mbind` is the general form of what `HierarchicalDistribution` provided: a primary measure, a transition kernel and a combination function (`merge`, `vcat`, `tuple`, `=>`), with densities, draws and transports (the Rosenblatt transform) derived structurally. The 275-line distribution and its `UnshapedHDist` flat view go away with it. `distbind` and `lbqintegral` are stable API, so they stay as deprecated wrappers for `mbind` resp. `mintegrate`/`mintegrate_exp`. Note that measures with binds inside declare no degrees of freedom, so `transform_function` sizes their standard target from the variate shape. Created by generative AI.
Working notes on what the branch removes, keeps and changes, the plan for variate shapes and ValueShapes, the conventions adopted from MeasureBase, the accelerator plan and the open questions. To be removed before the merge. Created by generative AI.
Follow-ups to building BAT's measures on MeasureBase, from running the test suite: MeasureBase fuses nested pushforwards into one pushforward under a function chain, so `has_uhc_support` walks the chain and lets the last step that determines the variate values decide; shaping and unshaping pass the question on. Measures with value-dependent variate sizes (binds) declare no degrees of freedom, and their variate shape can overcount them when a marginal is constant, so the size of a standard transport target comes from a test transport instead. `distprod` accepts the marginal specifications of a `NamedTupleDist` (distributions, intervals, arrays of those, constants), which the tutorial's prior needs. `eff_totalndof` (superseded by `MeasureBase.getdof`) and `near_neg_inf` (used only by the deleted pushforward density) have no users left. Created by generative AI.
`BispacedMeasure` stamps its transformed side with a hash of the transformation that produced it. MeasureBase's measures and transports compare by value but inherit the object-based `hash` fallback whenever they hold mutable data (a distribution's parameter arrays, a product measure's marginal storage), so two equal transformations got different stamps and the cached transformed representations of an `EvaluatedMeasure` were never reused. `BAT.transform_witness` hashes the structure of a transformation explicitly instead. A value-based `Base.hash` for measures in MeasureBase would make it unnecessary. Created by generative AI.
The tests of the removed duplicates go away: the distribution-measure wrapper, the power and weighted measures, BAT's standard uniform and normal distributions, `HierarchicalDistribution` and the 251-line test of BAT's own distribution transforms. `test_bat_pushfwd_measure.jl` becomes `test_pushfwd_measure.jl`, built on `transport_to`, `pushfwd` and `mbind`, and keeps its end-to-end sampling, mode-finding and integration checks. `test_bat_superpos_measure.jl` becomes `test_superpos_measure.jl` and keeps the numerics that matter (the log-density at ties, zero-mass components, sampling from the mixture), without asserting BAT-specific types. `test_measure_functions.jl` now covers what `distprod` builds and the two deprecated wrappers. Aqua's piracy check gets the MeasureBase types that BAT extends towards ValueShapes and Statistics, until that support moves into a ValueShapes extension of MeasureBase. NEWS gets a v6.0.0 section for the breaking changes, and the version is bumped accordingly. Created by generative AI.
Prior substitution replaces a prior of limited support by a standard measure of full support, so the posterior's zero-prior short-circuit can no longer keep the likelihood from being evaluated at points the original prior excluded. `BAT.SupportedDensity` carried the original support along, but evaluated the transport into that support in addition to the precomposed likelihood, and masked the result instead of branching, so the likelihood was still called there. It now holds the un-precomposed likelihood together with the support and the full map into it, evaluates that map once and branches on the support check, so the likelihood is only ever called at points of the original support. The posterior's short-circuit itself was an override of `DensityInterface.logdensityof`, which bypassed MeasureBase's point path with its shape checks as well as the batched path, where the likelihood was evaluated regardless of the prior. It is now MeasureBase's designated extension point `logdensityof_impl`, which both paths go through. A posterior also forwards the flat variate size of its prior, so that batched evaluation can tell variate from batch dimensions. Created by generative AI.
`MeasureBase.weightedmeasure(w, ::AbstractPosteriorMeasure)` returned a `PosteriorMeasure` with a rescaled likelihood instead of a `WeightedMeasure`, overriding a MeasureBase generic with a different representation for every caller, BAT's or not. The old BAT had a separate internal function for this, which is restored as `BAT._bat_weightedmeasure`: it falls back to `weightedmeasure` and keeps only BAT's own reweighting (auto-renormalization, reweighted evaluated measures, prior substitution of a reweighted measure) posterior-shaped. Created by generative AI.
`checked_logdensityof` let every non-finite log-density at a non-finite variate pass as a zero density. MeasureBase's convention only makes such variates yield `NaN`, an infinite density is a model error wherever it occurs, so only `NaN` escapes now. The output number type of a transport was taken to be that of the input variates, on the assumption that transports preserve precision. They don't: transporting `Float32` variates into a `Float64` measure yields `Float64`. The input number type is promoted with that of the target measure's variates instead, which is still cheap enough to sidestep the inference failure that the shortcut was there for. Also documents that measures without declared degrees of freedom are transformed on the evidence of a single test transport, drops the MeasureBase imports that nothing uses any more, moves the `_logaddexp` test to the utilities it belongs to, and records the migration's remaining behavior changes in NEWS.md. Created by generative AI.
Variate shapes of MeasureBase measures, unshaped measures, shapes applied to measures and the result shapes of transports now come from ValueShapes (0.11.8, MeasureBase extension), together with the conversion of NamedTupleDist, ConstValueDist and ReshapedDist to measures via asmeasure, so BAT's own copy of that support goes away. Created by generative AI.
The MCMC hot loop evaluated the target density of every walker on its own, via `checked_logdensityof.(target, X)`. `BAT.checked_logdensities(target, X)` hands the whole batch of walker positions to `MeasureBase.logdensities` instead and applies BAT's NaN/+Inf policy to the results afterwards: an `any` over a branch-free validity mask, and only where something is invalid does the per-variate check run and throw. The batched evaluation itself carries no checks and no try/catch, so it stays a single operation that measures with batched kernels can serve in one go. Walker positions live in `ArrayOfSimilarArrays`, whose flat storage MeasureBase fuses at the entry point, but broadcasting a transform over one yields a `VectorOfArrays`, which MeasureBase treats as ragged and evaluates variate by variate. The proposal step therefore copies the proposed positions into the chain state's buffer before evaluating them, and the proposal transitions are built as `VectorOfSimilarVectors`, so that the Hastings terms of all walkers are evaluated in one go as well. Densities of measures with fused batched kernels can differ from the per-variate results in the last ulp, so MCMC runs with a fixed seed may take a different path than before. Created by generative AI.
Without a batched kernel of its own a posterior measure falls back to MeasureBase's default, which maps the point kernel over the batch and so evaluates the prior variate by variate as well. `MeasureBase.batched_logdensityof_impl` for posterior measures evaluates the prior for the whole batch instead and the likelihood variate by variate (it has no batched form), only where the prior density is nonzero. The point kernel keeps that short-circuit and now shares its implementation. Created by generative AI.
The likelihood of a posterior with a substituted prior is evaluated only at finite transported variates, which replaces the support check of the original prior: MeasureBase's transports keep finite inputs finite, so only infinite inputs reach the mask, and an isfinite reduction costs nothing next to the likelihood, while insupport of hierarchical priors can cost as much as the density itself. Created by generative AI.
MeasureBase 0.15 builds on the static size tools of StaticThings 0.3. Created by generative AI.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Written by Claude (Fable 5.1, high effort) on behalf of the maintainer.
Work in progress. BAT built on the new MeasureBase (0.15, JuliaMath/MeasureBase.jl#182): BAT's own measure layer, distribution wrappers, standard distributions and transform implementation are removed in favor of MeasureBase's measures,
transport_toandmbind.PosteriorMeasurestays as a thinAbstractMeasuresubtype,lbqintegralanddistbindare deprecated.The approach, what goes and what stays, the plan for variate shapes and ValueShapes, the accelerator plan and the open questions are documented in
redesign.mdin the repository root. That file is kept up to date while the branch evolves and will be removed before the merge.Still to do before merging: the open questions in
redesign.md, the accelerator work (separate commits), NEWS, a version bump to 6.0.0.https://claude.ai/code/session_01L5yPndh2K6u14z3CwF9YXf