feat(atoms): Wave 4 — atom finding, GPU refinement, property maps - #85
feat(atoms): Wave 4 — atom finding, GPU refinement, property maps#85CSSFrancis wants to merge 2 commits into
Conversation
Division of labour, and why this wave is small: atomap owns the STRUCTURE (peak finding, sublattices, neighbours, zone axes, dumbbell pairing) because reimplementing it would diverge from published atomap results. SpyDE owns the REFINEMENT, because refining atom positions IS a batched 2-D gaussian fit and spyde.fitting already does the whole field in one Levenberg-Marquardt where atomap fits one atom at a time with scipy. Only find_atoms needs the atoms extra; refinement and the property maps are plain numpy/torch, so they are testable and usable without it. Correctness comes from ground truth: the new atom_lattice() generator knows exactly where every atom is, so refinement is scored against those positions rather than against atomap's answer. The full pipeline (find -> COM -> gaussian) recovers them to a MEDIAN error of 0.2 px, and refinement alone to better than 0.1 px worst-case. Tolerances are tight deliberately — a test accepting +/-1 px would pass with the refinement removed entirely. Details that are easy to get wrong and are pinned by tests: - A pinned atom (the red/green toggle, #76) is EXCLUDED from the fit, not fitted and discarded — otherwise the work is wasted and a diverging neighbour can still perturb the shared solve. - A fit that leaves its own box keeps its INPUT position. A wild coordinate is worse than an unrefined one. - Gaussian A is a VOLUME, so the amplitude seed is scaled by 2*pi*sigma^2; seeding it with the peak height starts every fit an order of magnitude low. - Ellipticity is max/min sigma, never sigma_x/sigma_y: the latter drops below 1 for an atom elongated along y and puts a spurious boundary wherever the elongation direction changes. - Displacement uses a LOCAL reference (the centroid of k neighbours), so it measures real distortion and is immune to drift or a tilted scan, which a globally-fitted ideal lattice would report as displacement. atom_lattice() also grows dumbbell / displacement / ellipticity knobs so the downstream maps have something real to recover instead of a uniform zero.
Many structures image as PAIRS of closely-spaced atoms. Treating each pair as two independent atoms throws away what the pair actually measures — the separation and orientation that carry polarisation, tilt and local strain. Follows atomap's workflow: estimate the dumbbell vector, pair the atoms, then measure. The vector is estimated from the data rather than typed in, since it depends on the projected structure and the scan rotation. The part worth reading is refine_pairs, which exists because writing the tests exposed a systematic error rather than a bug. Fitting each atom of a dumbbell INDEPENDENTLY biases its centre TOWARDS its partner: the partner's tail is signal a single gaussian can only explain by moving. The separation therefore comes out too small — and uniformly so, which makes it look like a calibration rather than an error. Measured on a synthetic dumbbell of 6.0 px with sigma 2.0, independent fits return 4.21 px, a 30% underestimate. Fitting both atoms in ONE box as one two-gaussian model recovers 6.0, because each atom now explains the other's tail instead of absorbing it. Every pair is still one batched call — the engine does not care that the model gained a component. This is a good advert for the engine: the fix is a different MODEL, not different code. Three tests pin it: that independent fitting shows the bias (and says so if it ever stops), that joint fitting recovers the truth, and that joint beats independent by at least 5x — so a regression in either path shows up as the gap closing from the wrong side. Also pinned, because each is silently plausible if wrong: - The vector estimate must not CANCEL. Each pair contributes +v from one atom and -v from the other, so averaging without folding onto a half-plane gives exactly zero, which looks like a legitimate answer. - Pairing must not claim an atom twice, or there are more pairs than atoms. - The angle is folded to a half-turn, or two identical dumbbells differ by pi depending on which atom was listed first and an angle map shows a boundary that is not there.
#78 dumbbell lattices — and a systematic error the tests exposedEstimate the dumbbell vector → pair the atoms → measure. The vector is estimated from the data rather than typed in, since it depends on the projected structure and the scan rotation. The part worth reading is Fitting each atom of a dumbbell independently biases its centre towards its partner — the partner's tail is signal a single gaussian can only explain by moving. The separation comes out too small, and uniformly so, which makes it look like a calibration rather than an error.
Fitting both atoms in one box as one two-gaussian model fixes it, because each atom now explains the other's tail instead of absorbing it. Every pair is still one batched call — the engine doesn't care that the model gained a component. This is a good advert for the engine, incidentally: the fix was a different model, not different code. Three tests pin it — that independent fitting shows the bias (and says so if that ever stops being true), that joint fitting recovers the truth, and that joint beats independent by ≥5×, so a regression in either path shows as the gap closing from the wrong side. Also pinned, because each is silently plausible if wrong
Wave 4's compute layer is now complete: #75, #77, #78, #79. 50 tests green. Still open here: #76 (the three atomap GUI functions as anyplotlib overlays) — UI, needs screenshots. |
Closes #75, #77, #79. Part of tracker #48.
Stacked on #82 (needs the 2-D fitting engine) — base retargets to
mainonce that lands.Division of labour
atomap owns the structure — peak finding, sublattices, neighbours, zone axes, dumbbell pairing — because reimplementing it would diverge from published atomap results.
SpyDE owns the refinement, because refining atom positions is a batched 2-D gaussian fit, and
spyde.fittingalready does the whole field in one Levenberg–Marquardt where atomap fits one atom at a time with scipy.Only
find_atomsneeds theatomsextra; refinement and the property maps are plain numpy/torch, so they're testable and usable without it.Accuracy, against ground truth
The new
atom_lattice()generator knows exactly where every atom is, so refinement is scored against those positions rather than against atomap's answer:Tolerances are tight deliberately — a test accepting ±1 px would pass with the refinement removed entirely.
Details pinned by tests because they're easy to get wrong
Ais a volume, so the amplitude seed is scaled by2πσ². Seeding with peak height starts every fit an order of magnitude low.Data
atom_lattice()grows dumbbell / displacement / ellipticity knobs so the downstream maps have something real to recover instead of a uniform zero. The grid is non-square and the ellipticity test requires the right-hand atoms to be measurably more elliptical than the left — a transposed or flipped result fails.25 tests, all green.
Still open in this wave
#76 (the three atomap GUI functions as anyplotlib overlays) and #78 (dumbbell lattices). #76 is UI — it gets verified by running the app and looking at pixels, which I've deliberately left for when you're around rather than doing unattended.