From 42e591e5eb77dacd61b5c8a274f617159bcd272f Mon Sep 17 00:00:00 2001 From: delchev Date: Thu, 20 Aug 2026 12:24:54 +0300 Subject: [PATCH] fix(intent): a settlement follows a re-filed payment, and a delayed create event can no longer shrink it (#6864, #6865) Two residues of #6846, found in its review: - A corrected MATCH column - the payment re-filed under another Customer - was invisible to the amount-based recompute (pot - allocated == 0), so the junction rows kept paying the OLD counterparty's invoices forever. The payment's match columns are now grouping keys, so its DAO publishes "-rekeyed" when one moves, and a third settlement listener releases everything and re-allocates from the STORE - both re-key notices run the same store-driven recompute, so their delivery order cannot matter and a redelivery converges. Emitted only for a LOCAL payment: a cross-model payment's DAO belongs to the owner model, and a projection has no repository to re-read - absent rather than silently dead. Closes #6864. - The create handler no longer releases. A create event is the FIRST word about a payment - it has nothing to take back - so a negative pot there can only be a DELAYED create racing a correction the updated handler already allocated, and releasing on the stale payload transiently undid the correction. The updated handler owns every shrink. Closes #6865. Also corrects the module guide's glue-collection note, which still cited the generateUtils.js twin removed by #6707. Covered by GlueSettlementRekeyTest (the third listener, and its deliberate absence for a cross-model payment), EdmIntentGeneratorTest (the match columns as the payment's grouping keys), and IntentEngineIT's settlement emission block (the store-driven re-key recompute, the release-all, and that the create handler carries no release). Co-Authored-By: Claude Fable 5 --- components/engine/engine-intent/CLAUDE.md | 2 +- .../intent/generator/GlueIntentGenerator.java | 16 +++ .../generator/edm/EdmIntentGenerator.java | 14 +++ .../generator/GlueSettlementRekeyTest.java | 99 +++++++++++++++++++ .../generator/edm/EdmIntentGeneratorTest.java | 53 ++++++++++ .../events/SettlementOnPayment.java.template | 30 +++++- .../integration/tests/api/IntentEngineIT.java | 18 ++++ 7 files changed, 228 insertions(+), 4 deletions(-) create mode 100644 components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/GlueSettlementRekeyTest.java diff --git a/components/engine/engine-intent/CLAUDE.md b/components/engine/engine-intent/CLAUDE.md index 6bb1eb83e34..97fbd0798aa 100644 --- a/components/engine/engine-intent/CLAUDE.md +++ b/components/engine/engine-intent/CLAUDE.md @@ -686,7 +686,7 @@ Implemented and generating annotated client-Java off the shared `EventBinding` / - **Editor renders the glue + outputs.** New `renderGlue()` "Glue & Outputs" mxGraph section with SAP-icon-badged cards edged to their entities (see the diagram section). `IntentEditorLoadsIT` asserts the `loanUpdated` card renders. - **Parser hardening.** A wrong-typed scalar (e.g. an unquoted brace recipient `to: {member.email}`, which YAML parses as an object) now surfaces as a clean `IntentValidationException` issue with a helpful message instead of a raw 500 Gson error that wedged the editor. `IntentParserTest` covers it. - **Externalized AI system prompt.** Moved from an inline string to `intent-assistant-guide.md` (classpath resource, fail-fast load), corrected to the full current schema incl. the glue catalog + `businessKey`/`businessKeyStrategy`, and restored the propose-the-whole-file tool contract the draft had dropped. -- **Settlements re-allocate a corrected payment (#6818).** The payment spread handler used to bind the payment's bare create topic only, so a payment booked for the wrong amount and corrected afterwards - or created incomplete and completed later - was never re-allocated and the invoice kept the original settled figure. It is now emitted once per bound payment event (create + `-updated`) from its own glue collection, **`settlementListeners`** - the `settlements` collection still drives the one-per-settlement `OnInvoice` delegate, and a second collection is what lets the two templates fan out differently while sharing one descriptor (`rollupEntry` copies it per class name + topic suffix, exactly as roll-ups and expansions do). Note the two traps a new glue collection carries: it needs a `case` in **both** `GlueGenerator` (Java) and `generateUtils.js` (JS) or it renders whole-model with raw `${...}` placeholders, and `GlueGenerator.copy` is a per-key allow-list, so a new descriptor key that is not listed there never reaches the template. The handler itself was already a recompute of the payment's *unallocated* balance (re-delivery is a no-op by construction); it now also **releases** the excess - newest allocation first, through the junction repository - when the payment is corrected below what it already covers, so the recompute converges in both directions. +- **Settlements re-allocate a corrected payment (#6818).** The payment spread handler used to bind the payment's bare create topic only, so a payment booked for the wrong amount and corrected afterwards - or created incomplete and completed later - was never re-allocated and the invoice kept the original settled figure. It is now emitted once per bound payment event (create + `-updated`) from its own glue collection, **`settlementListeners`** - the `settlements` collection still drives the one-per-settlement `OnInvoice` delegate, and a second collection is what lets the two templates fan out differently while sharing one descriptor (`rollupEntry` copies it per class name + topic suffix, exactly as roll-ups and expansions do). Note the two traps a new glue collection carries: it needs a `case` in `GlueGenerator` (Java - the only generator since #6707; the old `generateUtils.js` twin is gone) or it renders whole-model with raw `${...}` placeholders, and `GlueGenerator.copy` is a per-key allow-list, so a new descriptor key that is not listed there never reaches the template. The handler itself was already a recompute of the payment's *unallocated* balance (re-delivery is a no-op by construction); it now also **releases** the excess - newest allocation first, through the junction repository - when the payment is corrected below what it already covers, so the recompute converges in both directions. Two boundaries of that release: the CREATE handler never releases - a create event is the first word about a payment, so a negative pot there can only be a DELAYED create racing a correction the updated handler already allocated, and releasing on the stale payload would undo it (#6865). And a corrected MATCH column (the payment re-filed under another Customer) is invisible to the amount-based recompute (`pot - allocated == 0`), so the payment's match columns are grouping keys and a third listener on the payment's `-rekeyed` topic releases everything and re-allocates from the STORE - both re-key notices run the same store-driven recompute, so delivery order cannot matter; emitted only for a LOCAL payment, since a cross-model payment's DAO belongs to the owner model and a projection has no repository to re-read (#6864). - **CI runs on Corretto 24** (compile target stays 21); the integration-test fork gets `-Xmx6g`. (Root-level change; recorded here because it landed alongside the intent work.) **Cross-artefact field naming:** the `.form` control `model` (and control `id`) bind to the entity property, so they use `IntentNaming.pascalCase` to match the EDM property names (`loanedOn` -> `LoanedOn`). The `.report` references physical UPPER_SNAKE columns and humanized display aliases (no camelCase property identifiers), so it needs no PascalCasing. diff --git a/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java b/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java index 3846611abf6..08ec2f6977a 100644 --- a/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java +++ b/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java @@ -686,6 +686,14 @@ private static List> buildSettlementListeners(List> buildRollupsForTest(IntentModel model) { null); } + /** Test hook: build the {@code settlementListeners} glue collection without a repository. */ + static List> buildSettlementListenersForTest(IntentModel model) { + IntentGenerationContext context = + new IntentGenerationContext(model, "/" + model.getName(), model.getName(), "workspace", model.getName(), null); + return buildSettlementListeners(buildSettlements(model, IntentEntities.byName(model), IntentEntities.compositionParents(model), + IntentSettings.parse("{}"), context)); + } + /** Test hook: build the {@code waits} glue collection without a repository. */ static List> buildWaitsForTest(IntentModel model) { return buildWaits(model, IntentSettings.parse("{}")); diff --git a/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/edm/EdmIntentGenerator.java b/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/edm/EdmIntentGenerator.java index a2590c95e4f..d635b55b352 100644 --- a/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/edm/EdmIntentGenerator.java +++ b/components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/edm/EdmIntentGenerator.java @@ -51,6 +51,7 @@ import org.eclipse.dirigible.components.intent.model.RelatedIntent; import org.eclipse.dirigible.components.intent.model.RelationIntent; import org.eclipse.dirigible.components.intent.model.RollupIntent; +import org.eclipse.dirigible.components.intent.model.SettlementIntent; import org.eclipse.dirigible.components.intent.model.SlotsIntent; import org.eclipse.dirigible.components.intent.model.UsesIntent; import org.eclipse.dirigible.components.intent.model.UniqueIntent; @@ -634,6 +635,19 @@ else if (!extension && !dependent && !setting && !compositionParents.containsVal addGroupingKey(groupingKeys, seenGroupingKeys, rollup.getVia()); } } + // A settlement matches invoices by columns of the PAYMENT row (its Customer, its Company), + // so a correction that moves one of them re-targets the whole allocation - the settlement's + // rekey handler releases and re-allocates from the store. Without these keys the DAO never + // notices the move and the junction rows keep paying the OLD counterparty's invoices. + for (SettlementIntent settlement : model.getSettlements()) { + if (entity.getName() + .equals(settlement.getPayment()) + && settlement.getMatch() != null) { + for (String matchColumn : settlement.getMatch()) { + addGroupingKey(groupingKeys, seenGroupingKeys, matchColumn); + } + } + } if (!groupingKeys.isEmpty()) { entityMap.put("groupingKeys", groupingKeys); FieldIntent groupingPk = primaryKeyOf(entity); diff --git a/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/GlueSettlementRekeyTest.java b/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/GlueSettlementRekeyTest.java new file mode 100644 index 00000000000..c189b53f680 --- /dev/null +++ b/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/GlueSettlementRekeyTest.java @@ -0,0 +1,99 @@ +/* + * Copyright (c) 2010-2026 Eclipse Dirigible contributors + * + * All rights reserved. This program and the accompanying materials are made available under the + * terms of the Eclipse Public License v2.0 which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v20.html + * + * SPDX-FileCopyrightText: Eclipse Dirigible contributors SPDX-License-Identifier: EPL-2.0 + */ +package org.eclipse.dirigible.components.intent.generator; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.util.List; +import java.util.Map; + +import org.eclipse.dirigible.components.intent.parser.IntentParser; +import org.junit.jupiter.api.Test; + +/** + * A corrected settlement MATCH column re-targets the whole allocation: the payment's DAO must + * notice the move (the match columns are grouping keys) and the settlement must bind a rekey + * handler that releases and re-allocates from the store. Without both, the junction rows kept + * paying the OLD counterparty's invoices forever - the amount-based recompute sees + * {@code pot - allocated == 0} and no-ops. + */ +class GlueSettlementRekeyTest { + + private static final String YAML = """ + name: settle + entities: + - name: Invoice + fields: + - { name: id, type: integer, primaryKey: true, generated: true } + - { name: date, type: date } + - { name: total, type: decimal, precision: 18, scale: 2 } + - { name: paid, type: decimal, precision: 18, scale: 2 } + relations: + - { name: Customer, kind: manyToOne, to: Customer } + - name: Payment + fields: + - { name: id, type: integer, primaryKey: true, generated: true } + - { name: date, type: date } + - { name: amount, type: decimal, precision: 18, scale: 2, required: true } + relations: + - { name: Customer, kind: manyToOne, to: Customer } + - name: Customer + fields: + - { name: id, type: integer, primaryKey: true, generated: true } + - { name: name, type: string, required: true, length: 100 } + - name: InvoicePayment + fields: + - { name: id, type: integer, primaryKey: true, generated: true } + - { name: amount, type: decimal, precision: 18, scale: 2, required: true } + relations: + - { name: Invoice, kind: manyToOne, to: Invoice, composition: true, required: true } + - { name: Payment, kind: manyToOne, to: Payment, required: true } + settlements: + - { name: autoSettle, junction: InvoicePayment, invoice: Invoice, payment: Payment, + amount: amount, total: total, paid: paid, pot: amount, order: date, + match: [Customer] } + """; + + @Test + void aLocalPaymentGetsTheRekeyListenerBesideCreateAndUpdated() { + List> listeners = GlueIntentGenerator.buildSettlementListenersForTest(IntentParser.parse(YAML)); + + assertEquals(3, listeners.size(), "create, correction and re-key - the third is what re-targets a corrected match column"); + assertEquals(List.of("", "-updated", "-rekeyed"), listeners.stream() + .map(l -> String.valueOf(l.get("topicSuffix"))) + .toList()); + assertEquals(List.of("AutoSettleOnPayment", "AutoSettleOnPaymentUpdated", "AutoSettleOnPaymentRekeyed"), listeners.stream() + .map(l -> String.valueOf( + l.get("className"))) + .toList()); + } + + /** + * A cross-model payment's DAO is generated by the OWNER model, which knows nothing of this + * settlement - no "-rekeyed" is ever published for it here, and the store-driven re-allocation + * needs the payment's repository, which a projection does not have. The rekey listener is + * deliberately absent rather than silently dead. + */ + @Test + void aCrossModelPaymentGetsNoRekeyListener() { + // The cross-model shape is the junction's payment relation carrying model:. + String yaml = YAML.replace("name: settle\n", "name: settle\nuses:\n - { model: treasury, project: treasury }\n") + .replace("- { name: Payment, kind: manyToOne, to: Payment, required: true }", + "- { name: Payment, kind: manyToOne, to: Payment, required: true, model: treasury }"); + assertTrue(yaml.contains("model: treasury"), "the fixture surgery must actually make the payment cross-model"); + List> listeners = GlueIntentGenerator.buildSettlementListenersForTest(IntentParser.parse(yaml)); + + assertEquals(2, listeners.size(), "a cross-model payment binds create and correction only: " + listeners); + assertTrue(listeners.stream() + .noneMatch(l -> "-rekeyed".equals(l.get("topicSuffix")))); + } + +} diff --git a/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/edm/EdmIntentGeneratorTest.java b/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/edm/EdmIntentGeneratorTest.java index 58aa1fea1ae..ec4672c45fa 100644 --- a/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/edm/EdmIntentGeneratorTest.java +++ b/components/engine/engine-intent/src/test/java/org/eclipse/dirigible/components/intent/generator/edm/EdmIntentGeneratorTest.java @@ -821,6 +821,59 @@ void rollupChildCarriesItsParentFkAsAGroupingKey() { assertNull(entityByName(entities(model), "Customer").get("groupingKeys")); } + /** + * A settlement matches invoices by columns of the PAYMENT row, so a corrected match value (the + * payment re-filed under another Customer) moves the whole allocation between counterparties. The + * payment must track those columns as grouping keys - the DAO's "-rekeyed" publish is the only + * signal the settlement's re-key handler ever gets. + */ + @Test + @SuppressWarnings("unchecked") + void aSettlementsMatchColumnsAreThePaymentsGroupingKeys() { + String yaml = """ + name: settle + entities: + - name: Invoice + fields: + - { name: id, type: integer, primaryKey: true, generated: true } + - { name: date, type: date } + - { name: total, type: decimal, precision: 18, scale: 2 } + - { name: paid, type: decimal, precision: 18, scale: 2 } + relations: + - { name: Customer, kind: manyToOne, to: Customer } + - name: Payment + fields: + - { name: id, type: integer, primaryKey: true, generated: true } + - { name: date, type: date } + - { name: amount, type: decimal, precision: 18, scale: 2, required: true } + relations: + - { name: Customer, kind: manyToOne, to: Customer } + - name: Customer + fields: + - { name: id, type: integer, primaryKey: true, generated: true } + - { name: name, type: string, required: true, length: 100 } + - name: InvoicePayment + fields: + - { name: id, type: integer, primaryKey: true, generated: true } + - { name: amount, type: decimal, precision: 18, scale: 2, required: true } + relations: + - { name: Invoice, kind: manyToOne, to: Invoice, composition: true, required: true } + - { name: Payment, kind: manyToOne, to: Payment, required: true } + settlements: + - { name: autoSettle, junction: InvoicePayment, invoice: Invoice, payment: Payment, + amount: amount, total: total, paid: paid, pot: amount, order: date, + match: [Customer] } + """; + Map model = EdmIntentGenerator.buildModelJsonForTest(IntentParser.parse(yaml), "settle"); + List> keys = (List>) entityByName(entities(model), "Payment").get("groupingKeys"); + assertEquals(1, keys.size(), "the payment must track its settlement match column"); + assertEquals("Customer", keys.get(0) + .get("key")); + // The invoice side is matched by the RECORD's own column and never moves an allocation by + // itself - only the payment's match value re-targets the settlement. + assertNull(entityByName(entities(model), "Invoice").get("groupingKeys")); + } + /** * The two non-blocking guard outcomes. Both PERSIST the row and mark it instead of failing the * write: {@code task} stamps a boolean marker that the entity's process decision branches on (the diff --git a/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/SettlementOnPayment.java.template b/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/SettlementOnPayment.java.template index 629b757bac5..a7d97475204 100644 --- a/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/SettlementOnPayment.java.template +++ b/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/SettlementOnPayment.java.template @@ -15,15 +15,18 @@ import gen.${javaGenFolderName}.data.${invoiceJavaPerspective}.${invoiceEntity}R import gen.${javaGenFolderName}.data.${junctionJavaPerspective}.${junctionEntity}Entity; import gen.${javaGenFolderName}.data.${junctionJavaPerspective}.${junctionEntity}Repository; import gen.${paymentGenFolder}.data.${paymentJavaPerspective}.${paymentEntity}Entity; +#if($topicSuffix == "-rekeyed") +import gen.${paymentGenFolder}.data.${paymentJavaPerspective}.${paymentEntity}Repository; +#end /** - * Auto-settlement (on ${paymentEntity} #if($topicSuffix == "")create#{else}correction#end): allocates the payment across the payer's open + * Auto-settlement (on ${paymentEntity} #if($topicSuffix == "")create#elseif($topicSuffix == "-rekeyed")re-key#{else}correction#end): allocates the payment across the payer's open * ${invoiceEntity}s - oldest first, matching ${match} - creating ${junctionEntity} rows until the * payment is used up. The paid roll-up then updates each invoice's paid / balance / status. * * This is a RECOMPUTE of the payment's unallocated balance, not an append: it allocates only what is - * still unallocated, and releases the excess - newest allocation first - when the payment is corrected - * to less than what it already covers. That is what makes it safe to bind to the payment's correction + * still unallocated#if($topicSuffix != ""), and releases the excess - newest allocation first - when the payment is corrected + * to less than what it already covers#end. That is what makes it safe to bind to the payment's correction * event as well as to its create one, so a payment booked for the wrong amount, or completed after it * was created, is settled for the amount it actually carries. * @@ -51,10 +54,31 @@ public class ${className} implements MessageHandler { if (payment == null || payment.${paymentPk} == null || payment.${paymentPot} == null) { return; } +#if($topicSuffix == "-rekeyed") + // A grouping column of the payment moved - its match value was corrected - so the allocation + // is re-targeted WHOLESALE: release everything, then allocate against the payment as it now + // stands IN THE STORE. Both re-key notices (the previous row and the written one) run this + // same store-driven recompute, so their delivery order cannot matter, and a redelivery finds + // the same converged state. The payload is used only for the key. + ${paymentEntity}Entity stored = new ${paymentEntity}Repository().findById(payment.${paymentPk}); + if (stored == null || stored.${paymentPot} == null) { + return; + } + release(payment.${paymentPk}, allocated(payment.${paymentPk})); + payment = stored; +#end BigDecimal pot = payment.${paymentPot}.subtract(allocated(payment.${paymentPk})); if (pot.signum() < 0) { +#if($topicSuffix == "") + // A create event is the FIRST word about a payment - it has nothing to release. Reaching a + // negative pot here means this event was DELAYED past a correction the updated handler has + // already allocated; releasing on this stale payload would undo that correction until the + // next payment event happened to repair it. The updated handler owns every shrink. + return; +#else release(payment.${paymentPk}, pot.negate()); return; +#end } if (pot.signum() == 0) { return; diff --git a/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEngineIT.java b/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEngineIT.java index fc47ae00838..c68d5bcba72 100644 --- a/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEngineIT.java +++ b/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEngineIT.java @@ -1962,6 +1962,11 @@ void settlement_generates_on_payment_listener_and_on_invoice_delegate() { "it should create allocation rows through the junction repository (never the generic Store)"); assertTrue(onPayment.contains("return \"" + PROJECT + "-Payment-Payment\";"), "the create listener should bind the bare payment topic"); + // A create event is the FIRST word about a payment - it has nothing to release. A negative pot + // on this handler means the event was DELAYED past a correction the updated handler already + // allocated, and releasing on that stale payload would undo the correction (#6865). + assertFalse(onPayment.contains("release(payment.Id, pot.negate())"), + "the create listener must never release - the updated handler owns every shrink"); // A payment corrected after it was booked - or created incomplete and completed later - must be // re-allocated, so the same recompute is bound to the payment's update event too (#6818). @@ -1975,6 +1980,19 @@ void settlement_generates_on_payment_listener_and_on_invoice_delegate() { assertTrue(onPaymentUpdated.contains(".orderByDesc(\"Id\")") && onPaymentUpdated.contains("rows.delete(row)"), "the release should give back the newest allocations first, through the junction repository"); + // A corrected MATCH column (the payment re-filed under another Customer) re-targets the whole + // allocation: the payment's DAO publishes "-rekeyed" for the move (the match columns are + // grouping keys) and this third handler releases everything and re-allocates from the STORE - + // both re-key notices run the same store-driven recompute, so delivery order cannot matter + // (#6864). + String onPaymentRekeyed = contentOf("gen/events/settle/AutoSettleOnPaymentRekeyed.java"); + assertTrue(onPaymentRekeyed.contains("return \"" + PROJECT + "-Payment-Payment-rekeyed\";"), + "the re-key listener should bind the payment's -rekeyed topic"); + assertTrue(onPaymentRekeyed.contains("new PaymentRepository().findById(payment.Id)"), + "the re-key recompute must read the payment from the store, never trust the moved payload"); + assertTrue(onPaymentRekeyed.contains("release(payment.Id, allocated(payment.Id))"), + "the re-key recompute must release the whole allocation before re-allocating"); + String onInvoice = codeOf("gen/events/settle/AutoSettleOnInvoice.java"); assertTrue(onInvoice.contains("class AutoSettleOnInvoice implements JavaDelegate"), "the onInvoice settlement delegate should be generated");