test: deprecate aws lambda based integration tests because now we have e2e and the RIE based tests - #826
Merged
Conversation
joeyzhao2018
force-pushed
the
joey/deprecate-aws-integration-tests
branch
2 times, most recently
from
September 17, 2026 14:41
dd684d2 to
ff1df38
Compare
This comment has been minimized.
This comment has been minimized.
joeyzhao2018
changed the title
joeyzhao2018
marked this pull request as ready for review
September 17, 2026 15:47
lucaspimentel
approved these changes
Sep 17, 2026
Member
|
@DataDog review |
… serverless-e2e-tests) The integration_tests/ AWS suite is retired. Its coverage now lives in two places, each stronger at what it owns: - Behavior: integration_tests_local/ (docker/RIE, GitHub Actions, per PR) is now the behavior oracle — hermetic, contributor-runnable, and strictly stronger than the AWS suite at behavior (its goldens preserve span meta/metrics keys the AWS normalization stripped wholesale). After #821 and #823 it covers every variant the AWS suite pinned, including callback-style handlers, fetch, metrics-only mode, payload capture, and the manual-wrap no-init path. - Real-AWS-only signals: the four variants whose value IS the platform — esm (published layer zip artifact + ESM loader), sync-metrics (direct-API sink to the real intake), http-requests (_X_AMZN_TRACE_ID pass-through + real egress), status-code-500s (real API Gateway) — move to a dedicated integration-tests-residual suite in serverless-e2e-tests, which deploys against published/candidate layer ARNs and keeps the same snapshots and byte-identical normalization pipeline. Deleted here: serverless.yml, the six top-level handlers, all 450 snapshots, and scripts/run_integration_tests.sh (459 files). Kept: input_events/, parse-json.js, and container/ — the local suite references the first two and uses container/ as fixture; a new integration_tests/README.md documents this. CI: the GitLab `integration test` job and its sign/publish `needs` entries are removed (build-layer keeps CI_ENABLE_CONTAINER_IMAGE_BUILDS, which it uses to docker-build the layer). CONTRIBUTING.md now points contributors at the local suite. Release-gate note: the per-pipeline AWS gate on sign/publish goes away. The candidate-layer gate moves to the residual suite in serverless-e2e-tests — manual until the e2e release-gate automation lands (migration roadmap), so release runbooks should trigger it until then.
joeyzhao2018
force-pushed
the
joey/deprecate-aws-integration-tests
branch
from
September 17, 2026 16:08
a2ec9ac to
fcfac45
Compare
![datadog-prod-us1-5[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
The new contributor command does not run the same runtime matrix as CI. It fails on older hosts or uses the wrong tracer pack for Node 18 and Node 20.
🤖 Datadog Autotest · Commit a2ec9ac · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Remove all the aws-lambda based integration tests because we now use RIE based tests, which is much stronger covering more cases and we have e2e tests for all real lambda based test suites.
I also updated this repo's setting by adding a rule in the rulesets, which makes sure the RIE tests are passing for any updates to the main branch.
I don't know why the task "dd-gitlab/integration test" is still showing up in this PR's checks... checked with AI and it doesn't have a solid answer but saying probably it's from previous commits and should be gone once the PR gets merged.