Skip to content

feat(atoms): Wave 4 — atom finding, GPU refinement, property maps - #85

Open
CSSFrancis wants to merge 2 commits into
feat/0.3.0-wave0-wave1from
feat/0.3.0-wave4-atoms
Open

feat(atoms): Wave 4 — atom finding, GPU refinement, property maps#85
CSSFrancis wants to merge 2 commits into
feat/0.3.0-wave0-wave1from
feat/0.3.0-wave4-atoms

Conversation

@CSSFrancis

Copy link
Copy Markdown
Owner

Closes #75, #77, #79. Part of tracker #48.

Stacked on #82 (needs the 2-D fitting engine) — base retargets to main once that lands.

Division of labour

atomap owns the structure — peak finding, sublattices, neighbours, zone axes, dumbbell pairing — because reimplementing it would diverge from published atomap results.

SpyDE owns the refinement, because refining atom positions is a batched 2-D gaussian fit, and spyde.fitting already does the whole field in one Levenberg–Marquardt where atomap fits one atom at a time with scipy.

Only find_atoms needs the atoms extra; refinement and the property maps are plain numpy/torch, so they're testable and usable without it.

Accuracy, against ground truth

The new atom_lattice() generator knows exactly where every atom is, so refinement is scored against those positions rather than against atomap's answer:

median error
full pipeline (find → COM → gaussian) 0.2 px
refinement alone, worst case < 0.1 px

Tolerances are tight deliberately — a test accepting ±1 px would pass with the refinement removed entirely.

Details pinned by tests because they're easy to get wrong

  • A pinned atom is excluded from the fit, not fitted and then discarded. Otherwise the work is wasted and a diverging neighbour can still perturb the shared solve. (This is the red/green toggle from [4.2] The three atomap GUI functions as anyplotlib overlays #76.)
  • A fit that leaves its own box keeps its input position — a wild coordinate is worse than an unrefined one.
  • Gaussian A is a volume, so the amplitude seed is scaled by 2πσ². Seeding with peak height starts every fit an order of magnitude low.
  • Ellipticity is max/min σ, never σx/σy. The latter drops below 1 for an atom elongated along y, putting a spurious boundary wherever the elongation direction changes.
  • Displacement uses a local reference (centroid of k neighbours), so it measures real distortion and is immune to drift or a tilted scan — a globally-fitted ideal lattice reports both as displacement.

Data

atom_lattice() grows dumbbell / displacement / ellipticity knobs so the downstream maps have something real to recover instead of a uniform zero. The grid is non-square and the ellipticity test requires the right-hand atoms to be measurably more elliptical than the left — a transposed or flipped result fails.

25 tests, all green.

Still open in this wave

#76 (the three atomap GUI functions as anyplotlib overlays) and #78 (dumbbell lattices). #76 is UI — it gets verified by running the app and looking at pixels, which I've deliberately left for when you're around rather than doing unattended.

Division of labour, and why this wave is small: atomap owns the STRUCTURE
(peak finding, sublattices, neighbours, zone axes, dumbbell pairing) because
reimplementing it would diverge from published atomap results. SpyDE owns the
REFINEMENT, because refining atom positions IS a batched 2-D gaussian fit and
spyde.fitting already does the whole field in one Levenberg-Marquardt where
atomap fits one atom at a time with scipy.

Only find_atoms needs the atoms extra; refinement and the property maps are
plain numpy/torch, so they are testable and usable without it.

Correctness comes from ground truth: the new atom_lattice() generator knows
exactly where every atom is, so refinement is scored against those positions
rather than against atomap's answer. The full pipeline (find -> COM -> gaussian)
recovers them to a MEDIAN error of 0.2 px, and refinement alone to better than
0.1 px worst-case. Tolerances are tight deliberately — a test accepting +/-1 px
would pass with the refinement removed entirely.

Details that are easy to get wrong and are pinned by tests:

- A pinned atom (the red/green toggle, #76) is EXCLUDED from the fit, not
  fitted and discarded — otherwise the work is wasted and a diverging neighbour
  can still perturb the shared solve.
- A fit that leaves its own box keeps its INPUT position. A wild coordinate is
  worse than an unrefined one.
- Gaussian A is a VOLUME, so the amplitude seed is scaled by 2*pi*sigma^2;
  seeding it with the peak height starts every fit an order of magnitude low.
- Ellipticity is max/min sigma, never sigma_x/sigma_y: the latter drops below 1
  for an atom elongated along y and puts a spurious boundary wherever the
  elongation direction changes.
- Displacement uses a LOCAL reference (the centroid of k neighbours), so it
  measures real distortion and is immune to drift or a tilted scan, which a
  globally-fitted ideal lattice would report as displacement.

atom_lattice() also grows dumbbell / displacement / ellipticity knobs so the
downstream maps have something real to recover instead of a uniform zero.
Many structures image as PAIRS of closely-spaced atoms. Treating each pair as
two independent atoms throws away what the pair actually measures — the
separation and orientation that carry polarisation, tilt and local strain.

Follows atomap's workflow: estimate the dumbbell vector, pair the atoms, then
measure. The vector is estimated from the data rather than typed in, since it
depends on the projected structure and the scan rotation.

The part worth reading is refine_pairs, which exists because writing the tests
exposed a systematic error rather than a bug. Fitting each atom of a dumbbell
INDEPENDENTLY biases its centre TOWARDS its partner: the partner's tail is
signal a single gaussian can only explain by moving. The separation therefore
comes out too small — and uniformly so, which makes it look like a calibration
rather than an error. Measured on a synthetic dumbbell of 6.0 px with
sigma 2.0, independent fits return 4.21 px, a 30% underestimate.

Fitting both atoms in ONE box as one two-gaussian model recovers 6.0, because
each atom now explains the other's tail instead of absorbing it. Every pair is
still one batched call — the engine does not care that the model gained a
component. This is a good advert for the engine: the fix is a different MODEL,
not different code.

Three tests pin it: that independent fitting shows the bias (and says so if it
ever stops), that joint fitting recovers the truth, and that joint beats
independent by at least 5x — so a regression in either path shows up as the gap
closing from the wrong side.

Also pinned, because each is silently plausible if wrong:

- The vector estimate must not CANCEL. Each pair contributes +v from one atom
  and -v from the other, so averaging without folding onto a half-plane gives
  exactly zero, which looks like a legitimate answer.
- Pairing must not claim an atom twice, or there are more pairs than atoms.
- The angle is folded to a half-turn, or two identical dumbbells differ by pi
  depending on which atom was listed first and an angle map shows a boundary
  that is not there.
@CSSFrancis

Copy link
Copy Markdown
Owner Author

#78 dumbbell lattices — and a systematic error the tests exposed

Estimate the dumbbell vector → pair the atoms → measure. The vector is estimated from the data rather than typed in, since it depends on the projected structure and the scan rotation.

The part worth reading is refine_pairs. Writing the end-to-end test surfaced not a bug but a systematic measurement error:

Fitting each atom of a dumbbell independently biases its centre towards its partner — the partner's tail is signal a single gaussian can only explain by moving. The separation comes out too small, and uniformly so, which makes it look like a calibration rather than an error.

measured separation (true = 6.0 px, σ = 2.0)
independent fits 4.21 px — a 30% underestimate
joint pair fit 6.0 px

Fitting both atoms in one box as one two-gaussian model fixes it, because each atom now explains the other's tail instead of absorbing it. Every pair is still one batched call — the engine doesn't care that the model gained a component.

This is a good advert for the engine, incidentally: the fix was a different model, not different code.

Three tests pin it — that independent fitting shows the bias (and says so if that ever stops being true), that joint fitting recovers the truth, and that joint beats independent by ≥5×, so a regression in either path shows as the gap closing from the wrong side.

Also pinned, because each is silently plausible if wrong

  • The vector estimate must not cancel. Each pair contributes +v from one atom and −v from the other, so averaging without folding onto a half-plane gives exactly zero — which looks like a legitimate answer.
  • Pairing must not claim an atom twice, or there are more pairs than atoms.
  • The angle is folded to a half-turn, or two identical dumbbells differ by π depending on which atom was listed first, and an angle map shows a boundary that isn't there.

Wave 4's compute layer is now complete: #75, #77, #78, #79. 50 tests green.

Still open here: #76 (the three atomap GUI functions as anyplotlib overlays) — UI, needs screenshots.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant