MODEXPW-635 - Expose template tokens in EDIFACT email export by markusweigelt · Pull Request #737 · folio-org/mod-data-export-worker

markusweigelt · 2026-05-27T15:23:32Z

MODEXPW-635
- Expose template tokens in EDIFACT email export

Purpose

Make order, order-line, and vendor-organization data available as {{token}} placeholders in the email templates rendered for the EDIFACT email export. Before this change, SendToEmailTasklet invoked TemplateEngineClient.processTemplate with templateId + lang + outputFormat but no context payload, so templates couldn't reference any order data — only the static template body reached the recipient.

Approach

Ship the worker a structured context object that mod-template-engine substitutes against the configured template.

1. Context payload shape

  {
    "createdAt": "...",
    "organization": { name, primaryAddress: { addressLine1, city, zipCode, country } },
    "orders": [
      {
        "order": {
          poNumber, orderType,
          metadata: { createdByUser: { id, firstName, lastName, fullName } },
          shipTo: { id, address },
          billTo: { id, address }
        },
        "orderLines": [
          { "orderLine": { poLineNumber, titleOrPackage, publisher, publicationDate, edition, rush,
                           contributors: [ { contributor, contributorNameType: { id, name } } ],
                           details: { productIds: [ { productId, qualifier, productIdType: { id, name } } ] },
                           cost: { listUnitPrice, listUnitPriceElectronic,
                                   quantityPhysical, quantityElectronic,
                                   poLineEstimatedPrice, currency },
                           fundDistribution: [ { code } ],
                           vendorDetail: { instructions } } }
        ]
      }
    ]
  }

2. Token surface exposed to templates

createdAt
organization.*: name, primaryAddress.{addressLine1, city, zipCode, country}
order.*: poNumber, orderType, metadata.createdByUser.{id, firstName, lastName, fullName}, shipTo.{id, address}, billTo.{id, address}
orderLine.*: poLineNumber, titleOrPackage, publisher, publicationDate, edition, rush, contributors[].{contributor, contributorNameType.{id, name}}, details.productIds[].{productId, qualifier, productIdType.{id, name}}, cost.{listUnitPrice, listUnitPriceElectronic, quantityPhysical, quantityElectronic, poLineEstimatedPrice, currency}, fundDistribution[].code, vendorDetail.instructions

3. Resolutions performed by context mapper

OrderEmailContextMapper.buildContext(orders) is the single entry point. It walks each CompositePurchaseOrder, pairs each PoLine with its own parent order (so multi-order exports don't leak the first order's poNumber onto later lines), and resolves the following FOLIO lookups before handoff:

createdAt — generated at build time in ISO-8601 UTC.
order.metadata.createdByUser — resolved from metadata.createdByUserId via new UserService.getUserContext(...) into a UserContext {id, firstName, lastName, fullName}; fullName is "firstName lastName" trimmed, falling back to username.
order.shipTo / order.billTo — address UUID resolved to a TenantAddressContext {id, address} via ConfigurationService.getTenantAddress(...).
orderLine.contributors[].contributorNameType.name — UUID → type name (e.g. Personal name) via cached ContributorNameTypeService.getContributorNameTypeName(uuid).
orderLine.details.productIds[].productIdType.name — UUID → type name (e.g. ISBN) via cached IdentifierTypeService.getIdentifierTypeName(uuid).
orderLine.fundDistribution[] — exposes only the fund code, taken as-is from the PO line (fundId is not resolved; expenseClassId, distributionType, and value are not exposed).
organization — resolved from the order vendor via OrganizationsService; primary address picked with a strict isPrimary == true filter.
Null-safe throughout: missing leaf strings fall back to "", quantities to 0, lists to [], and unresolved sub-objects (Cost, VendorDetail, metadata, vendor, addresses) to empty contexts.

Additional information

Example plain-text template body of mod-template-engine exercising every token:

Created at: {{createdAt}}

Organization: {{organization.name}}
  {{organization.primaryAddress.addressLine1}}
  {{organization.primaryAddress.city}}, {{organization.primaryAddress.zipCode}}
  {{organization.primaryAddress.country}}

{{#orders}}
==================================================
PO Number:  {{order.poNumber}}
Order Type: {{order.orderType}}
Created by: {{order.metadata.createdByUser.fullName}} ({{order.metadata.createdByUser.firstName}} {{order.metadata.createdByUser.lastName}}, id={{order.metadata.createdByUser.id}})

Ship To: {{order.shipTo.address}}   (id={{order.shipTo.id}})
Bill To: {{order.billTo.address}}   (id={{order.billTo.id}})

Order Lines:
{{#orderLines}}
  --------------------------------------------------
  PO Line:        {{orderLine.poLineNumber}}
  Title/Package:  {{orderLine.titleOrPackage}}
  Publisher:      {{orderLine.publisher}}
  Publication:    {{orderLine.publicationDate}}
  Edition:        {{orderLine.edition}}
  Rush:           {{orderLine.rush}}

  Contributors:
  {{#orderLine.contributors}}
    - {{contributor}} [{{contributorNameType.name}} / {{contributorNameType.id}}]
  {{/orderLine.contributors}}

  Product IDs:
  {{#orderLine.details.productIds}}
    - {{productId}} {{qualifier}} [{{productIdType.name}} / {{productIdType.id}}]
  {{/orderLine.details.productIds}}

  Cost:
    List Unit Price:            {{orderLine.cost.listUnitPrice}}
    List Unit Price (Electr.):  {{orderLine.cost.listUnitPriceElectronic}}
    Quantity Physical:          {{orderLine.cost.quantityPhysical}}
    Quantity Electronic:        {{orderLine.cost.quantityElectronic}}
    PO Line Estimated Price:    {{orderLine.cost.poLineEstimatedPrice}}
    Currency:                   {{orderLine.cost.currency}}

  Fund Distribution:
  {{#orderLine.fundDistribution}}
    - {{code}}
  {{/orderLine.fundDistribution}}

  Vendor Instructions: {{orderLine.vendorDetail.instructions}}
{{/orderLines}}
{{/orders}}

Depends on

PR #732 - MODEXPW-625 - Enable Email Delivery for EDIFACT Export Jobs

…ar URL for Feign routing

mod-template-engine returns 400 when outputFormat does not match the format registered in the template. The claims template requires text/html.

Read outputFormat from template-engine response meta and forward it to EmailEntity so the email is sent in the same format the template was rendered in (e.g. text/html).

Wrap templateEngineClient and emailClient calls in EdifactException, derive attachment content type from fileFormat, extract buildAttachment and sendEmail helper methods, and add unit/integration tests for the tasklet and its decider.

Remove template context from TemplateProcessingRequest for now since required template variables are not yet known; mod-template-engine treats a missing context as an empty object. Add ORDERING + EMAIL test coverage.

Migrate EmailClient and TemplateEngineClient from @FeignClient to Spring 6 @HttpExchange. Update Spring Batch import paths to new package structure. Fix SendToEmailTaskletTest to use JobOperatorTestUtils and correct mock types.

@bean

…iguration Both clients use @HttpExchange and require explicit @bean registration via HttpServiceProxyFactory — they were missing, causing a NoSuchBeanDefinitionException when sendToEmailStep attempted to inject EmailClient.

@RequestBody

Spring HttpServiceProxyFactory requires @RequestBody on POST parameters to resolve them as the request body — unlike Feign which inferred this from unannotated arguments.

…lization RestClient uses Jackson 3 (tools.jackson) which cannot deserialize into the abstract com.fasterxml.jackson.databind.JsonNode (Jackson 2). Return String instead and parse with ediObjectMapper in the tasklet.

…sponse DTO Jackson 3 (RestClient) cannot deserialize JSON into String or the abstract JsonNode (Jackson 2). A concrete DTO is the correct and consistent approach, matching how all other clients in this module handle responses.

Removed the erroneous CLAIMING integration type check; email delivery is driven exclusively by transmissionMethod == EMAIL regardless of integration type. Also added a comment in EdifactExportJobConfig summarising the decider conditions for all three optional steps.

Drop the unused JsonNode import and ADDRESS constant, and route both cacheable methods through a private fetchTenantAddress helper so the @Cacheable getTenantAddress is no longer self-invoked via this.

Cover getUserContext: resolved name, username fallback when names are blank or personal is absent, and empty context on null response or client exception.

markusweigelt · 2026-06-29T13:22:20Z

@markusweigelt please fix the merge conflicts first before we continue reviewing.

@SerhiiNosko I hope you're getting through the heat well. 🥵 Besides merging master, we also optimized the actual changes.

Skip null source entries before mapping in the shared list helper to avoid NPEs. Also drop the obsolete estimatedPrice field from the documented order-line payload in README.

@log4j2

Replace the manual LogManager.getLogger() field with Lombok @log4j2 to match the other acquisitions services in this branch.

Instant.now().toString() emits a variable number of fractional digits depending on clock resolution, breaking the required millisecond format. Truncate to millis and add a regression test guarding the pattern.

Let ContributorNameTypeService return the raw cached result and handle lookup failures in OrderEmailContextMapper, so a failed type resolution degrades to an empty name without breaking the email context build.

markusweigelt · 2026-07-02T11:48:59Z

@SerhiiNosko Thx for the review of this hugh PR and your approval! Can I merge this PR? There was still a question regarding caching. I would merge the Spring PR in the same step.

SerhiiNosko · 2026-07-02T11:53:56Z

@SerhiiNosko Thx for the review of this hugh PR and your approval! Can I merge this PR? There was still a question regarding caching. I would merge the Spring PR in the same step.

by the way, locally did you check functionality with eureka-cli?

markusweigelt · 2026-07-02T11:58:38Z

by the way, locally did you check functionality with eureka-cli?

Yes, we have a development system based on the Eureka CLI, and we have already tested the placeholder replacement there. We haven’t checked the permissions in the PR folio-org/mod-data-export-spring#380 yet - I’ll take care of that right away. But this should only be a formality.

SerhiiNosko · 2026-07-02T12:42:55Z

by the way, locally did you check functionality with eureka-cli?

Yes, we have a development system based on the Eureka CLI, and we have already tested the placeholder replacement there. We haven’t checked the permissions in the PR folio-org/mod-data-export-spring#380 yet - I’ll take care of that right away. But this should only be a formality.

lets merge then, also for future you can create story to cover this functionality wit karate tests, karate tests are executed every night and do real testing of services without any mocks and always help us to find new bugs. In this case we will make sure that this newly implemented functionality always work as expected.
Here is folder where our existing karate tests live: https://github.com/folio-org/folio-integration-tests/tree/master/acquisitions/src/main/resources/thunderjet/mod-data-export-spring

sonarqubecloud · 2026-07-02T12:59:32Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
96.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

markusweigelt · 2026-07-03T09:14:36Z

also for future you can create story to cover this functionality wit karate tests, karate tests are executed every night and do real testing of services without any mocks and always help us to find new bugs. In this case we will make sure that this newly implemented functionality always work as expected.
Here is folder where our existing karate tests live: https://github.com/folio-org/folio-integration-tests/tree/master/acquisitions/src/main/resources/thunderjet/mod-data-export-spring

Thanks for the note. Integration tests are of course even better.

How does this work with this story - do you mean creating a Jira issue as a story type for this? Regarding the implementation of the integration tests, are you providing them based on the story, or are we supposed to deliver them? (We’re still fairly new here, hence these questions.)

lets merge then

I just tested the workflow via Spring again, and it works. The test was run in our environment; in general, we have already been doing this via Spring.

Could you please give me the approval on the PR folio-org/mod-data-export-spring#380 in the Spring module? Then I would merge both PRs afterwards.

SerhiiNosko · 2026-07-06T08:49:11Z

also for future you can create story to cover this functionality wit karate tests, karate tests are executed every night and do real testing of services without any mocks and always help us to find new bugs. In this case we will make sure that this newly implemented functionality always work as expected.
Here is folder where our existing karate tests live: https://github.com/folio-org/folio-integration-tests/tree/master/acquisitions/src/main/resources/thunderjet/mod-data-export-spring

Thanks for the note. Integration tests are of course even better.

How does this work with this story - do you mean creating a Jira issue as a story type for this? Regarding the implementation of the integration tests, are you providing them based on the story, or are we supposed to deliver them? (We’re still fairly new here, hence these questions.)

lets merge then

I just tested the workflow via Spring again, and it works. The test was run in our environment; in general, we have already been doing this via Spring.

Could you please give me the approval on the PR folio-org/mod-data-export-spring#380 in the Spring module? Then I would merge both PRs afterwards.

Approved, so we can merge.
About karate tests - here is my config, I also run with using eureka-cli

There is export profile in eureka-cli, so you can use it after deploying combined-native application.
About karate tests - as you prefer, we commonly implement in scope of the same ticket, but it is already huge story, so you can create anoter Jira ticket to implement them. Karate always help us to find bugs and make sure that functionality is working as expected.

markusweigelt · 2026-07-06T09:02:56Z

Thanks for the approvals. 👍🏻

About karate tests - as you prefer, we commonly implement in scope of the same ticket, but it is already huge story, so you can create anoter Jira ticket to implement them. Karate always help us to find bugs and make sure that functionality is working as expected.

I’ll take a look at these integration tests and provide appropriate tests for our implementations.

markusweigelt added 30 commits March 11, 2026 13:09

Update submodule

a01605b

Add implementation to send emails

4dff439

Update submodule

b4d94e7

Improvement to compile module

055fa21

Improvement to compile module

1dbf08c

Improvement to compile module

eeff6fd

Improve email implementation

8cdaf43

Override Kafka-propagated x-okapi-url with worker's own Eureka sidec…

3e32a52

…ar URL for Feign routing

add logging

fae19f8

add configuration

c64c9b5

Update Client

d403eba

Update client config

cce9aef

Update emailclient

98ec7c9

Add template request

3b860bd

Update template engline

b11579b

fix(template-engine): change default outputFormat to text/html

0348c22

mod-template-engine returns 400 when outputFormat does not match the format registered in the template. The claims template requires text/html.

propagate template outputFormat to email entity

39425f0

Read outputFormat from template-engine response meta and forward it to EmailEntity so the email is sent in the same format the template was rendered in (e.g. text/html).

Add SendToEmailTasklet error handling and tests

d4d70e7

Wrap templateEngineClient and emailClient calls in EdifactException, derive attachment content type from fileFormat, extract buildAttachment and sendEmail helper methods, and add unit/integration tests for the tasklet and its decider.

Support email sending for ORDERING integration type

f8b55a9

Remove template context from TemplateProcessingRequest for now since required template variables are not yet known; mod-template-engine treats a missing context as an empty object. Add ORDERING + EMAIL test coverage.

Merge master into send-mail

a0a85cd

Add @RequestBody to EmailClient and TemplateEngineClient parameters

1f66e3d

Spring HttpServiceProxyFactory requires @RequestBody on POST parameters to resolve them as the request body — unlike Feign which inferred this from unannotated arguments.

Update SendToEmailTaskletTest mock to use TemplateProcessingResponse DTO

690e88d

Fix missing newline at end of file in several source files

db458f6

Update folio-export-common submodule branch to MODEXPW-625

050fb92

Remove stale Feign client debug logging from application.yml

de25a6c

markusweigelt force-pushed the MODEXPW-635 branch from 9d54d76 to 4c3fa7e Compare June 29, 2026 10:44

markusweigelt added 3 commits June 29, 2026 14:33

Fix ConfigurationService SonarCloud findings

e3d8639

Drop the unused JsonNode import and ADDRESS constant, and route both cacheable methods through a private fetchTenantAddress helper so the @Cacheable getTenantAddress is no longer self-invoked via this.

Add UserServiceTest for order email user context

6249170

Cover getUserContext: resolved name, username fallback when names are blank or personal is absent, and empty context on null response or client exception.

Remove unused imports from UserServiceTest

d0df2ec

markusweigelt requested a review from SerhiiNosko June 29, 2026 13:18