Skip to content

Prevent the usage of xfail in tests files - #7538

Merged
cbeauchesne merged 2 commits into
mainfrom
cbeauchesne/prevent-xfail-use
Aug 19, 2026
Merged

cbeauchesne merged 2 commits into
mainfrom
cbeauchesne/prevent-xfail-use

Conversation

@cbeauchesne

Copy link
Copy Markdown
Collaborator

Motivation

system-tests offers a semantics way to mark expected failure. In #7408, we plan to prevent the usage of bare @xfail marker, so we need to prevent the usage of it elsewhere in the code.

Changes

Change the skip logic for cassette generation.

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

tests/ai_guard/conftest.py                                              @DataDog/k9-ai-guard @DataDog/system-tests-core
tests/integration_frameworks/conftest.py                                @DataDog/system-tests-core
tests/parametric/test_llm_observability/conftest.py                     @DataDog/ml-observability @DataDog/system-tests-core

@cbeauchesne
cbeauchesne marked this pull request as ready for review August 19, 2026 08:48
@cbeauchesne
cbeauchesne requested review from a team as code owners August 19, 2026 08:48

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d66129f973

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread tests/integration_frameworks/conftest.py
Comment thread tests/integration_frameworks/conftest.py
Comment thread tests/integration_frameworks/conftest.py Outdated
Comment thread tests/parametric/test_llm_observability/conftest.py Outdated
@cbeauchesne

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 05d3d8e109

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Aug 19, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 65 jobs - 65 passed on retry View in Datadog

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 05d3d8e | Docs | View more details | Give us feedback!

@cbeauchesne
cbeauchesne enabled auto-merge (squash) August 19, 2026 12:22
@cbeauchesne
cbeauchesne merged commit ead0f63 into main Aug 19, 2026
815 of 826 checks passed
@cbeauchesne
cbeauchesne deleted the cbeauchesne/prevent-xfail-use branch August 19, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants