Skip to content

fix: validate instantiated job models with the template's extension context - #362

Open
wyongzhi wants to merge 2 commits into
OpenJobDescription:mainlinefrom
wyongzhi:fix/name-length-context-none
Open

fix: validate instantiated job models with the template's extension context#362
wyongzhi wants to merge 2 commits into
OpenJobDescription:mainlinefrom
wyongzhi:fix/name-length-context-none

Conversation

@wyongzhi

@wyongzhi wyongzhi commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Fixes: no linked issue (reported directly; see the repro below).

What was the problem/requirement? (What/Why)

create_job re-validates the models it instantiates, but did so without a parsing context. instantiate_model built each target model with a bare target_model(**fields), so inside the models' @field_validators info.context was None. The FEATURE_BUNDLE_1 length validators introduced in #258 are written as 512 if context and "FEATURE_BUNDLE_1" in context.extensions else 128 (or else 64), so with no context they fall back to the base limits and reject values that decode_job_template had already accepted under the extension:

Field Accepted at decode (FEATURE_BUNDLE_1) Rejected at create_job
Job.name up to 512 "String should have at most 128 characters"
Environment.name up to 512 "name must be at most 64 characters long"
EmbeddedFileText.name up to 512 "name must be at most 64 characters long"
EmbeddedFileText.filename up to 256 "String must be at most 64 characters long"

Repro (before this change): decode the template below with supported_extensions=[FEATURE_BUNDLE_1], then create_job(job_template=..., job_parameter_values={}).

specificationVersion: jobtemplate-2023-09
extensions: [FEATURE_BUNDLE_1]
name: "JJJJ…(512 chars)"
steps:
  - name: s
    script:
      embeddedFiles:
        - {name: Run, type: TEXT, data: "echo hi", filename: run.sh}
      actions: {onRun: {command: echo}}

NameIdentifierLengthMixin (#318) already returns early when no context is present, which is why step names and parameter names were unaffected; the four validators above predate that pattern.

What was the solution? (How)

instantiate_model now seeds one parsing context per call from the root template's declared extensions (model.model_parsing_context_type(supported_extensions=model.extensions or [])) and threads it through the recursion; the final construction becomes target_model.model_validate(fields, context=context) instead of target_model(**fields). Public signature unchanged. The per-field TypeAdapter.validate_python calls stay context-free (nothing there reads info.context; nested instances are not re-validated). Models that do not bind model_parsing_context_type (internal test fixtures) still get context=None. The four validators in _model.py are untouched; two comments there are updated.

Why not skip the check when the context is missing: JobName limits apply "after the format string has been resolved" (2023-09 Template Schemas, §1.1.1). A short {{Param.X}} name can resolve to 129+ characters at job creation and must still be rejected without FEATURE_BUNDLE_1; the existing test_fails_to_instantiate (v0/v1) pins this and passes unchanged. openjd-rs does the same resolved-name re-check at job creation (create_job/mod.rs, create_job_rejects_interpolated_name_exceeding_128_chars). Seeding from the declared extensions matches what decode narrowed context.extensions to.

Open question: should the per-field TypeAdapter calls receive the same context? Left out since nothing reads it; happy to add for symmetry.

What is the impact of this change?

  • Templates declaring FEATURE_BUNDLE_1 can now use the raised limits through create_job; templates without it behave as before (a 129-char resolved name is still rejected).
  • Every context-gated validator on the target models now runs at instantiation with the declared extension set instead of being skipped. That set equals what decode narrowed to, so each check reproduces its decode-time result; the full suite shows no other outcome change.
  • Direct construction of models without a context is unchanged. Calling instantiate_model (exported only from openjd.model._internal) directly on a sub-model now validates with an empty extension set rather than none, so mixin-guarded name validators apply the base limit there; the sole in-tree caller passes the JobTemplate root.
  • Public API unchanged.

How was this change tested?

  • New TestCreateJobPreservesFeatureBundle1Lengths in test/openjd/model_v0/v2023_09/test_feature_bundle_1.py (30 tests, real decode -> create_job, no mocks):
    • each of the four fields at its FEATURE_BUNDLE_1 ceiling survives create_job, plus one round trip with all four at the ceiling; together with the resolved 512-char job name below these are the six tests that failed before the fix, with the messages in the table above;
    • ceiling + 1 rejected at decode under FEATURE_BUNDLE_1; base limits (128/129, 64/65) rejected at decode without the extension;
    • resolved job name via {{Param.N}}: 128 accepted / 129 rejected without the extension, 512 accepted / 513 rejected with it;
    • static type ceilings without a context (TypeAdapter(JobName), direct Environment / EmbeddedFileText construction at ceiling + 1);
    • filename path-separator rejection both under FEATURE_BUNDLE_1 decode and without a context.
  • hatch run lint clean; hatch run test: 5825 passed, 24 skipped, 3 xfailed (baseline 5795 passed; +30 new tests).
  • OpenJD conformance runner (conformance-tests/run_openjd_cli_tests.py, openjd-cli 0.7.7 installed against this checkout): 2023-09/FEATURE_BUNDLE_1/jobs/* 12 passed / 0 failed (3 Windows-only skipped), 2023-09/base/jobs/1.1.1--* 3/0, 2023-09/FEATURE_BUNDLE_1/job_templates/* 41/0, plus the three new substitution fixtures from the companion openjd-specifications PR 3/0; the invalid cases reject with "at most 128" / "at most 512".
  • The context=None branch for model classes that bind no parsing-context type is exercised by the existing test/openjd/model_v0/_internal/test_create_job.py (its BaseModelForTesting fixtures are revision-agnostic and do not bind one).

Companion PR (separate repo, not required for this one): OpenJobDescription/openjd-specifications#184 adds three conformance fixtures that cross the job-name length boundary via parameter substitution. The existing boundary fixtures are all literal names, so the suite did not cover the "after the format string has been resolved" clause.

Was this change documented?

No user-facing documentation change; this restores behaviour the schema already specifies. The two comments in _model.py that described instantiation as context-free are updated; docstrings for the changed helpers are updated in _create_job.py.

Is this a breaking change?

No. Public API and signatures are unchanged.

Does this change impact security?

No. The change adds a validation context to an in-memory model instantiation; it does not create or modify files, directories, or trust boundaries. No threat model update needed.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…ontext

create_job instantiated the target models with a bare constructor, so the
FEATURE_BUNDLE_1 length validators on Job.name, Environment.name,
EmbeddedFileText.name and EmbeddedFileText.filename ran without a parsing
context and fell back to the base limits, rejecting values that decode had
already accepted under the extension.

instantiate_model now seeds a ModelParsingContext from the root template's
declared extensions and constructs each target model with
model_validate(..., context=...). Resolved values (the job name) are still
checked against the extension-aware limit, matching the schema text and the
Rust implementation; literal fields keep the result decode already reached.
Models whose class binds no parsing-context type are constructed without a
context, exactly as before.

The two comments in _model.py that described instantiation as running
without a context are updated to match; no validator logic changes.

Signed-off-by: Yongzhi Wei <276409147+wyongzhi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants