Add calibrated k-factor quantification of real AlSi10Mg 5 kV EDS spectrum - #11
Add calibrated k-factor quantification of real AlSi10Mg 5 kV EDS spectrum#11sgbaird wants to merge 6 commits into
Conversation
…trum Co-authored-by: Ronnie Guymon <244881888+ronnie-guymon@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@claude noting that edison should definitely be available now, in case there were things that needed to be run before (#1 (comment)) |
|
Claude finished @sgbaird's task in 27m 12s —— View job Edison ran successfully — the deferred issue #1 literature task is done and committed
The one Edison item still outstanding from before the key went live was the original issue #1 question (July 16): how are EDS parameters decided — not "consult the standards," but how were the standards' values chosen? That session got a 422 from an empty API key and promised a re-run. This is that re-run, enriched with this PR's live case (Mg Kα at 0.5–1 wt% in AlSi10Mg, 5 kV, SDD). Committed artifacts
Where each "standard" number actually comes from (Edison's synthesis, condensed)
For your exact case (Mg Kα 1.254 keV at 0.5–1 wt%, 5 kV): overvoltage is ample (U₀ ≈ 4), so excitation is not the limit — background modeling, the Al Kα neighbor, large low-energy absorption corrections, and surface carbon are. The literature-backed prescription: several 10⁵ to 10⁶ total counts (the naive 10⁴ rule is not enough at trace level), conservative dead time, a moderate fixed process time, replicate acquisitions — and standards-based quantification, since early standardless testing put 95% of results only within ±25% relative, unusable at 0.5 wt%. That independently reinforces this PR's k-factor-calibration approach and the DTSA-II route from the earlier discussion. Key commands and status• branch |
|
@claude also spot check that part about bulk corrections with eXSpy (see https://hyperspy.org/exspy/search.html?q=eds+bulk# for example) |
|
Claude finished @sgbaird's task in 7m 34s —— View job Spot check result: the bulk-corrections statements hold up against eXSpy 0.3.2
I took "that part about bulk corrections" to be the claims in the issue #1 write-up that this PR's script is built on. I verified each against the eXSpy EDS user guide and, more strictly, against the installed eXSpy 0.3.2 source — a grep of the source is a superset of the docs search you linked, since the docs are generated from the docstrings.
On the search page you linked: it's client-side JS, so I checked what actually contains "bulk". In the EDS portion of eXSpy it appears only in the interaction-volume utilities — One imprecision found (now fixed in the script header)The write-up said the calibrated k-factors are "valid for any other spectrum taken at the same kV/detector/geometry and fitted with the same recipe." For a bulk sample that list is missing a fourth condition: composition near the calibrant. True Cliff–Lorimer k-factors are composition-independent only in the thin-film limit; effective k-factors calibrated on bulk AlSi10Mg embed that specific matrix's ZAF effects, which change with composition. That is fine for comparing other AlSi10Mg builds (the intended use) and increasingly wrong as composition departs from the calibrant. The header of Key commands and statusNote: the Edison-availability comment is being handled by its own job run; it pushed --- • branch |
Co-authored-by: Sterling G. Baird <45469701+sgbaird@users.noreply.github.com>
Co-authored-by: Sterling G. Baird <45469701+sgbaird@users.noreply.github.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@claude from what you were saying here #1 (comment) "Where eXSpy still earns its keep in your workflow: it's the scriptable peak/background fitting and net-intensity extraction front end (and batch/map processing) — the numbers you then hand to DTSA-II or CalcZAF, which do the physics. If you'd rather do everything in one tool, DTSA-II alone covers both steps (it reads your .msa natively), and dropping eXSpy is a perfectly reasonable decision.", it made it seem oike DTSA-II can also do the scriptable peak/background fitting since you said DTSA-II covers both steps. So why not just used DTSA-II? Also in resopnse to #1 (comment) I have java 25, and it still doesn't work. I get the message The system cannot find the path specified. When I run just "C:\Users\rongu\Downloads\dtsa\DTSA-II.exe", it does open the screen for a split second before closing still |
|
Claude finished @ronnie-guymon's task in 16m 7s —— View job DTSA-II alone is a legitimate choice for your goal — and your launch failure is two commands away from a diagnosis
1. "Why not just use DTSA-II?" — you can, and for defensible wt% it's the better toolThat sentence meant exactly what you read: DTSA-II covers both steps, and dropping eXSpy is reasonable. The distinction I glossed over is what "scriptable" looks like in each:
2. Decoding the two symptoms (from the actual Polaris build config, not guesswork)I pulled the launcher and build configuration from the
3. Run this and paste back what it printsFrom a Command Prompt: cd /d C:\Users\rongu\Downloads\dtsa
dir /bThe listing is diagnostic on its own:
Aside: the deferred Edison literature task finishedWhile this ran, the earlier job's Edison LITERATURE_HIGH task (how the canonical EDS acquisition-parameter recommendations were originally derived) completed, and its artifacts are committed on this branch: Key commands run: • branch |
…were derived Deferred from the original issue #1 request (July 16), which could not run because the EDISON_PLATFORM_API_KEY secret was empty at the time. Task 2a23935c-f686-488e-a0a6-0faada1cb90f completed with status success. Co-authored-by: Sterling G. Baird <45469701+sgbaird@users.noreply.github.com>
|
@claude here's the files I see .installationinformation Here's the results I get |
|
Claude encountered an error after 1s —— View job I'll analyze this and get back to you. |
try this again |
|
Claude encountered an error after 1s —— View job I'll analyze this and get back to you. |
@claude Here's the results I get |
|
Claude encountered an error after 1s —— View job I'll analyze this and get back to you. |
|
@claude are you there? |
|
Claude encountered an error after 1s —— View job I'll analyze this and get back to you. |
|
Here's what I had to run in the command prompt in order to get DTSA-II to open: |
|
@claude+opus using the k-ratios from the EDS report, I got the Mg wt% to be about 0.9% as well. Correct me if I'm wrong, but that seems to be pretty close to the range that commercial AlSi10Mg is connected to have. It is definitely better than the around 1.5 wt% we had measured before. Is there a way to get the k-ratio directly from DTSA-II? I don't think so, and that is what I would have to use CalcZAF for? Becuase eXSpy doesn't return k-ratios? I would rather not use eXSpy if it only gives k-factors instead and is TEM centered, I want to be certain about the numbers I use and not have them be estimates or have to go through a conversion if possible. I would just upload the .msa file and use CalcZAF and use the k-ratios it gives me for the wt% input needed in DTSA-II. Help me understand the functions I would need from CalcZAF in order to get these wt%. Also, is the probe current 3.2nA that I uploaded correct? I know that's what the beam current was, I have just never seen "probe current" before. Screen.Recording.2026-08-21.115836.mp4 |
|
Claude finished @ronnie-guymon's task in 14m 6s —— View job DTSA-II does give you k-ratios directly — you picked the one wizard branch that asks you to supply them
I decoded your screen recording frame-by-frame, so the answers below are against what you actually did, not a generic workflow. 1. The 0.9 wt% number: better, but not yet in specI have to correct the premise. Standard AlSi10Mg is Mg 0.20–0.45 wt%, Si 9.00–11.00 wt% — identical limits in DIN EN 1706 / EN AC-43000 and ASTM F3318, per the Nikon SLM Solutions MDS (p. 2), and the same 0.20–0.45 in the EOS and Carpenter/AMS7018 sheets. So 0.90 wt% is roughly double the top of the range, not inside it. 1.5 → 0.9 is real progress; it isn't convergence. Your Si also lands at 11.15–11.33 wt%, just over the 11.00 ceiling. Mg high and Si high and Al low is a coherent pattern, not three independent flukes. Before chasing physics: pull the certificate of analysis for that powder lot / part. The CoA is the ground truth here, the spec range is only a bound. If the CoA says 0.4, you have a measurement bias; if it says 0.8, you're done. 2. Yes — DTSA-II gives k-ratios directly. You used the other branch.Your video shows the Quantification Alien dialog on "Specify the standards" → "Specify lines and k-ratios", typing
The first one is the default ( What it costs you: one standard spectrum per element, acquired on the same Octane Plus at the same kV with a known dose. Pure Al, pure Si, pure Mg, and something for O and C. 3. You do not need CalcZAFThe Quantification Alien k-ratio branch is CalcZAF's core function, already inside DTSA-II. Your result panel proves it: And CalcZAF could not do the step you actually want anyway — it has no spectrum reader and no peak deconvolution. Its File menu opens
So if you ever do want it, the button is the first one. Its only real value to you is as an independent check — it exposes many alternative ZAF/φ(ρz) formulations and MAC tables, where DTSA-II locked you into XPP + Chantler. That's a nice-to-have, not a dependency. 4. eXSpy: your instinct is rightConfirmed earlier in this PR against eXSpy 0.3.2 — no ZAF, no φ(ρz), and 5. "Probe current" — same thing as beam current, and only the ratio is usedProbe current = the electron current landing on the specimen (Faraday cup / picoammeter), in nA. NIST says "probe" because it's the current in the electron probe. DTSA-II pairs it with live time as probe dose (nA·s) to put unknown and standard on the same footing. In the wizard you used, the arithmetic is one line (L978): final double kr = krm.get(el).doubleValue() * rI.get(el).doubleValue() / kI.get(el).doubleValue();
// k-ratio you typed × I(standards panel) ÷ I(k-ratio panel)So 6. Two real bugs in the run, and the one that explains your total(a) Oxygen standard is That single error accounts for your missing total: An analytical total of 99.5% is what a self-consistent analysis should give. Fix: use pure-element materials for every row, or use (b) Live time mismatch. DTSA-II's properties panel shows Live time 30 s for 7. Why 0.9 is still 2× — and what would actually move itDTSA-II (XPP, modern MACs) got 0.900 and eZAF got 0.937 from the same k-ratios: agreement to 4% relative. The matrix correction is not your problem. No amount of re-running ZAF will move 0.9 to 0.4. The error is in the k-ratios themselves, which means peak deconvolution and standardless intensity scaling. Three things point the same way, and one dominates:
Ranked by value per hour:
Commands and verificationeZAF table read off the recording, used for every number above:
No files changed — this is analysis only. Also noting your |
@claude+opus in response to this, I'm thinking the best thing to do might be just to make another polished sample and then try. @gage-erickson got a vacuum chamber we can store the samples in, we will just need to use the pump in the polishing lab. The first time we ran EDS on this sample we did use 15kv and ended up getting 1.64 wt%. There also seems to be double the counts in the first EDS run, I think I remember doing a 15 minute scan where as the one we just did was only 7 minutes, 64 count vs 128 count I think. Basically, I don't think increasing voltage to 15keV would do anything if there is still oxygen contamination. Then we can worry about segregating the Al tail counts and the Mg peak counts once we have better quality data? |
|
Claude finished @ronnie-guymon's task in 11m 15s —— View job Re-polishing is the right next move — but your 15 kV screenshot changes the diagnosis, and it doesn't say what you think it says
1. You already ran the experiment I proposed, and it came back negativeLast comment I said: "Re-acquire at 15–20 kV. If Mg drops toward 0.3–0.4, the 5 kV result was surface-driven. Cheap, and it's a clean discriminator." You'd already done it, and Mg went up (0.94 → 1.64), not down. So that item is spent — scratch it off the list. Working in net-intensity ratios (the closest thing to raw data in the two reports, and immune to how each run was normalized):
O and Mg move up together; Si moves the other way. eZAF's entire job is to turn those raw ratios into a kV-independent answer for a homogeneous sample — and it didn't: O disagrees by 4.8×, Mg by 1.8× between the two runs. That's not a matrix-correction subtlety, that's the two datasets describing different material. 2. The catch: your 15 kV numbers are internally implausible, so don't anchor on 1.64 eitherO = 8.10 wt% cannot be a bulk composition. Bulk oxygen in AlSi10Mg powder runs 500–1250 ppm (0.05–0.125 wt%) — measured by inert-gas fusion across virgin and 30-month-recycled powder in Raza et al. 2021, Materials & Design 198, 109358. Your 15 kV value is 65–160× that. Your 5 kV value (1.68 wt%) is 13–34× — high, but in the range a surface film explains. 8.10 wt% is not. Three things make the 15 kV run the weaker dataset:
That's also the direct answer to your instinct: you're right that 15 kV isn't the fix, but for a stronger reason than oxygen. Low kV is inherently the more accurate choice for light elements because the absorption correction — the dominant error amplifier — is far smaller. Going up in kV to escape the surface trades a surface problem for a bigger absorption problem. Keep 5 kV for the number; use higher kV only as a depth probe, run head-to-head on the same spot (§6). 3. "Double the counts" — and why counts aren't your problemLive time was 460.8 s (15 kV) vs 327.7 s (5 kV), only 1.4×; the Al net is 2.2× higher mostly because 15 kV excites Al harder, not because the scan was longer. If the 15 min / 7 min wall-clock is right, the implied dead times are: 49% is out of the regime quantitative EDS is calibrated for. The literature review committed on this branch ( But the headline is: more counts cannot help you. Your Mg Error% was 2.43 at 15 kV = ±0.040 wt%. The gap from 1.64 down to the 0.45 spec ceiling is 30σ of counting error. No scan length touches a 30σ systematic. "Better quality data" has to mean cleaner and better-referenced, not longer. On "64 vs 128": I can't tell which APEX setting that is from the report alone (map frames? sweeps?) — it's recorded in the 4. Where the surface hypothesis actually stands — with numbersTwo things pull in opposite directions, so here's the arithmetic rather than a hand-wave. Against a pure surface film: a thin film's apparent concentration must fall with kV, because the sampled mass grows as roughly E₀^1.67. From 5 → 15 kV that predicts a 6× drop in the O signal. You measured a 2.5× rise. A surface film alone cannot do that. For a surface film (at 5 kV, where the data is trustworthy): the native oxide on Al is 2–4 nm, forms in air within milliseconds, and is self-limiting — you cannot polish it off in air (Evertsson et al., Appl. Surf. Sci. 2015). Folding that into the 0.42 µm interaction volume at 5 kV:
Measured: 1.68 wt%. So your 5 kV oxygen is about what an ordinary 3–10 nm native/hydrated oxide gives. That's the floor you can't polish below. And that film is Mg-enriched. Raza et al. found Mg²⁺ concentrated at the top of the powder's surface oxide as MgO or MgAl₂O₄ spinel, growing with powder reuse — while Si is absent from the oxide scale. Carry that through:
The excess you need to explain at 5 kV is 0.59 wt% (0.94 − 0.35). A Mg-enriched oxide of realistic thickness lands squarely in that band. So re-polishing is a bigger lever than I credited last time — your instinct is good. The part it can't explain: your Si is also high (11.15 vs an 11.00 ceiling), and Si is not in the oxide. Two of three excesses point at surface chemistry; the third points at standardless quantification. That's the case for doing both fixes, not one. 5. The polish + vacuum plan — yes, with three practical notesPolishing media matter more than storage. Colloidal silica adds Si and O; alumina suspension adds Al and O — both contaminate exactly what you're measuring, and both embed in soft Al. Finish with diamond (1 µm → 0.25 µm) in a non-aqueous / alcohol-based lubricant, ethanol rinse, N₂ blow-dry, no water (water drives oxide hydration and thickening). Storage buys you preservation, not removal. Vacuum/desiccator storage stops the 2–4 nm film from thickening and hydrating over days and cuts adsorbed hydrocarbon. It cannot remove a film that re-forms in milliseconds. If you want a genuinely oxide-free surface, that's broad-ion-beam / cross-section polishing (Gatan Ilion, Hitachi IM4000, JEOL CP) — worth asking around for. A plasma clean immediately before loading is the highest-yield cheap step for carbon; prefer a gentle Ar/H₂ or downstream air plasma over a hard O₂ plasma on an Al alloy. 6. Don't defer the Al-tail work — it's about 10 minutes in the same session
Sequencing them costs you a whole extra microscope session for no benefit. The tail work needs exactly one thing: a pure Al standard spectrum from your Octane Plus at the same kV, same process time, same probe current. That's a couple of minutes of scope time — and standards have to share a session with the unknown anyway (same detector state, same calibration), so acquiring them later is strictly worse than acquiring them alongside. With Al + Si + Mg (or MgO) standards in hand, take DTSA-II's first radio button — "Determine the composition of an 'unknown' spectrum by MLLSQ fitting to standards", the default you switched off last time. It fits your 7. The cheapest way to end the argumentNeither of these needs a microscope:
Physics also says your sample should read low in Mg, not high: LPBF melt-pool peak temperature (about 1900 K) exceeds Mg's boiling point (1363 K), so Mg preferentially evaporates and as-built parts trend below the feedstock (Springer, doi:10.1007/s13632-020-00659-w). Measuring 2–4× above the ceiling runs against the process physics — one more vote for measurement bias over real composition. Spec confirmed again from the Nikon SLM MDS: Mg 0.20–0.45, Si 9.00–11.00, identical under DIN EN 1706 and ASTM F3318. Next-session checklist (about 45 min of scope time)
Step 5 is the controlled version of the accidental experiment you already ran — same spot, same day, same dead time, same element list — so it actually answers "surface or not." Two things worth checking on your end
Checked and dismissed: the Si escape peak of the Al pile-up peakAt 49% dead time the Al Kα sum peak (2×1.4867 = 2.973 keV) is real, and its Si escape peak lands at 1.233 keV — 20 eV from Mg Kα at 1.2536 keV, i.e. buried inside the Mg peak. Sounds alarming. But the sum-peak fraction (order 1% of parent at high dead time) times the Si escape probability near 3 keV (order 1%) puts it at order 1 count against a 427-count excess. Not your problem — recorded so nobody chases it. No files changed — analysis only. Separately, Sources: Raza et al. 2021, Materials & Design 198 (open PDF) · Evertsson et al., Appl. Surf. Sci. 2015 — native oxide thickness · Nikon SLM AlSi10Mg MDS · Understanding the LPBF of AlSi10Mg · this branch's EDS-parameter literature review |
|
@claude+opus the powder had never been recycled and the sample was not repolished. How do I get standards for Al, Si, and Mg, and what even are the standards? Like what info does it contain? Is it just how many counts there are for each pure element? I tried finding files I could use to upload to DTSA-II to try and find wt%, but wasn't able to. Can you find some through a reliable source from Edison? |
|
Claude finished @ronnie-guymon's task in 30m 47s —— View job You can't download standards — DTSA-II refuses them in code. But two of the three you need cost nothing, and the answer PDF is already on your disk.
I answered from three sources: NIST's own DTSA-II/EPQ source code (what the software actually enforces), Ritchie's 0. Your two facts move the diagnosis toward standards, not polishingNot repolished is the bigger one. Last time I said this single fact changes the interpretation completely, and it does — against the plan. The 15 kV and 5 kV runs were the same surface, so nothing about the sample changed between them; only kV, dead time, and the element list did. That means the 15 kV O = 8.10 wt% is a measurement artifact, not evidence of a dirtier sample, and re-polishing cannot explain the 8.10 → 1.68 difference. Polishing is still worth doing — the 5 kV O of 1.68 wt% is a real 3–10 nm oxide + carbon film — but it is no longer the thing standing between you and a defensible number. Virgin powder cuts the same way. The Mg-enrichment-in-surface-oxide mechanism I leaned on (Raza et al.) was measured on recycled powder; their virgin powder had no detectable surface Mg²⁺. Your powder is virgin, so the mechanism I used to explain your 0.59 wt% Mg excess is at its weakest possible strength. Both facts push the same direction: the excess Mg is most likely instrumental, not real, and not surface. Which is exactly what standards fix. Your instinct to prioritize standards over another polish is right. 1. What a standard actually is — no, it is not just countsI pulled this from NIST's own code rather than describing it generically. DTSA-II will reject a spectrum as a standard unless all four of these are present (
So a standard is: counts + which detector produced them + at what kV + from what certified material. The counts alone are meaningless — they only become a standard when paired with a known composition and a dose you can normalize against. And there is a second kind of spectrum you're missing, which is the one that actually fixes your problem. DTSA-II distinguishes a standard from a reference:
That distinction is the whole ballgame for you. DTSA-II automatically demands a reference whenever two elements' regions of interest intersect ( Ritchie's own worked example of how much this matters (K240 glass,
Mg went from −11% to 0% by adding references. Same software, same physics, same sample. File format: a DTSA-II standard bundle is a 2. Why you couldn't find files to download — DTSA-II actively blocks themThis is not a "best practice." It is a hard exception in the loader ( if (!(zfh.DetectorName.equals(guid) || zfh.DetectorName.equals(det.getName())))
throw new EPQException("The standard does not appear to have been collected on the same detector as the unknown.");If you had found a The physics behind the refusal, from the Edison run: detector-to-detector differences in FWHM (122–150 eV at Mn Kα), window transmission (Moxtek AP3 vs AP5 differ in thickness and rib geometry; a Be window kills everything below about 1 keV), Si dead-layer thickness, and incomplete charge collection all reshape peaks — and the incomplete-charge-collection tail is the exact artifact you're fighting. A borrowed standard would import someone else's tail. Newbury & Ritchie do describe a "remote standards" scheme using another instrument's library with efficiency corrections, but they note no dose calibration is attempted, so the analytical total loses all validity and you must normalize — throwing away the single best diagnostic you have. Not a route I'd take when the goal is confidence in the number. What DTSA-II does ship is easy to mistake for a spectrum library:
Names of things to buy. Zero spectra. 3. How to actually get Al, Si and Mg standardsTwo of the three are free and you probably have them in the building.
Mount the Al and Si pieces in the same puck as your AlSi10Mg and polish them together, in the same session, with the same media. That gives identical surface prep on standard and unknown, which Newbury & Ritchie call out as a requirement at low kV, and it costs an afternoon. For Mg, the literature is unanimous: don't use Mg metal. Newbury & Ritchie's 2024 5 keV study used MgO for Mg, Al₂O₃ for Al, SiO₂ for Si. Pure Mg grows a surface MgO film in air and under the beam, which violates the homogeneity the k-ratio assumes. Verified purchase options:
The single best value here is not on that list. NIST SRM 470 (K412 glass) contains O–Mg–Al–Si–Ca–Fe with certified values, and Newbury & Ritchie used it in their 5 keV challenge set. Run it as an unknown and you get an independent, certified answer to "does my instrument read Mg correctly at 5 kV in an Al–Si matrix?" That settles your argument without touching the AlSi10Mg. Simulated standards — the option I'd steer you away from, with a number. 4. The two documents you already haveBoth shipped into
Also there: 5. Workflow once you have the materials
What this buys: Newbury & Ritchie (2024) ran this exact protocol at E₀ = 5 keV — 263 measurements, 39 elements, 113 materials — and got >98% within ±5% relative, 82% within ±2%. Against the roughly ±25% relative you're living with now, that is the difference between arguing about your number and reporting it. 6. The free experiment you can run before buying anythingYou don't need an Mg standard to test the Al-tail hypothesis — only to convert it to wt%. So:
If the residual Mg peak shrinks toward the 0.20–0.45 wt% band, you have your answer for the cost of a scope hour, and the MgO purchase becomes a formality to get the final number. Standards must be acquired in the same session as the unknown anyway (same detector state, same calibration), so pair this with whatever you run next rather than making it a separate trip. What I verified, and what I didn'tVerified in NIST source: the four standard requirements and their error strings, the detector-GUID refusal, the ROI-intersection reference rule, the Not verified: Astimex, Geller, Ted Pella, MAC and P&H current prices and catalogue IDs — get quotes. Edison also flagged that it could not confirm specific catalogue identifiers for those four suppliers from the literature, and I've kept that caveat rather than papering over it. Committed artifacts (
|
2 participants
Adds scripts/eds_kfactor_quant.py — a tested eXSpy workflow on the real 5 kV Apreo sum spectrum from issue #1: variance-floored Poisson-weighted model fit, bounded peak areas, Currie 3-sigma detection check, and k-factor calibration against a known composition. Includes the spectrum CSV and fit figure.
Addresses issue #1.
Generated with Claude Code