chore: simplify worker runtime and Docker dependency automation

[fqjony]

Summary

• remove worker-owned auth and deployment layers so provider auth stays external
• route entrypoint and CLI env/config re-apply through the shared runtime config path
• add opt-in redacted runtime JSON output and built-in default config files
• reorganize docs around CLI, config, secrets, services, deployment, and references
• add the scheduled/manual Docker dependency updater workflow powered by Copilot CLI
• merge the generated Docker dependency upgrade from chore(deps): docker dependency upgrade #129 into this release branch

Breaking Changes

• worker auth and the worker-owned provider login modules are removed. Consumers must establish AWS, Azure, or Google Cloud auth through the host platform, provider-native CLI, workload identity, mounted credentials, or child-image/runbook logic before resolving worker secrets.
• worker.yaml no longer uses the old merged-config path (/etc/worker/worker.merged.yaml). Runtime config now loads a mounted user config from /home/udx/.config/worker/worker.yaml when present, otherwise the built-in image default at /etc/worker/worker.yaml; consumers that read or mount the merged file need to update.
• Secrets are written into the restricted worker environment file instead of a separate secrets file. Scripts that inspect the old secrets file or expect worker env status to report separate secret-file counts need to use worker env show/worker env show --format json instead.
• Worker deployment templates and docs (deploy.yml, deploy-gcp.yml, docs/deploy/**, and the deploy-image-override example) are removed from this repo. Deployment now belongs to Docker, Kubernetes, CI/CD, or the external deployment tool that runs the image.
• Repository test assets moved from src/tests to test; local scripts or workflow path filters that mount or watch the old test path need to point at test.

Docker Dependency Changelog

• AZURE_CLI_VERSION: 2.85.0 -> 2.87.0
• PIP_VERSION: 26.0.1 -> 26.1.2
• GCLOUD_VERSION: 565.0.0 -> 571.0.0
• curl: 8.14.1-2ubuntu1.2 -> 8.14.1-2ubuntu1.3
• jq: 1.8.1-3ubuntu1 -> 1.8.1-3ubuntu1.1
• vim: 2:9.1.0967-1ubuntu6.2 -> 2:9.1.0967-1ubuntu6.5

Automation Changes

• add udx-automation / dependency upgrade workflow with config, analyze, and apply jobs
• keep repo-specific dependency hints in ci/configs/docker-dependency-probe.yaml
• keep Copilot update instructions in ci/prompts/docker-dependency-updater.md
• create automated dependency PRs as chore(deps): docker dependency upgrade
• request review from udx/worker and enable squash auto-merge when branch protection is satisfied

Validation

• bash -n lib/env_handler.sh lib/cli/env.sh lib/cli/config.sh lib/cli/service.sh lib/runtime_output.sh bin/entrypoint.sh test/modules/10_config.sh test/modules/20_env.sh
• worker env reload / worker config apply smoke test with temporary config paths
• Docker dependency updater workflow test completed successfully and produced chore(deps): docker dependency upgrade #129
• Docker Ops manual run for the generated dependency branch completed successfully
• Current chore: simplify worker runtime and Docker dependency automation #127 GitHub checks are passing, including docker-ops / build-and-scan
• make test blocked locally because Docker daemon socket was unavailable at /Users/jonyfq/.docker/run/docker.sock during the earlier runtime validation

[Copilot]

Pull request overview

This PR refactors the worker image runtime so that cloud provider authentication and deployment concerns are explicitly external, while keeping in-container config/env/secrets resolution and process supervision. It also reorganizes tests and documentation to match the new boundaries, and adds CI automation configuration for Docker dependency pin upgrades.

Changes:

• Remove worker-owned auth/cleanup layers and associated docs/tests; keep secret reference resolution after auth already exists.
• Route entrypoint + CLI “re-apply” flows through a shared configure_environment path; add opt-in runtime JSON output and built-in default configs.
• Restructure tests (src/tests → test) and reorganize docs around CLI/config/secrets/services/deployment/reference materials.

Reviewed changes

Copilot reviewed 63 out of 65 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/test_helpers.sh	Add shared test output helpers (colors/symbols).
test/modules/60_health.sh	Update helper source path to new test directory.
test/modules/50_sbom.sh	Update helper source path to new test directory.
test/modules/40_service.sh	Update helper source path to new test directory.
test/modules/20_env.sh	Update helper source path; add worker env reload test.
test/modules/10_config.sh	Update helper source path; add worker config apply test.
test/main.sh	Update test root path from /home/udx/tests → /home/udx/test.
src/tests/modules/30_auth.sh	Remove auth module tests (auth no longer owned by worker).
src/examples/simple-service/README.md	Update docs link to new services doc location.
src/examples/README.md	Remove deploy-image-override example reference.
src/examples/deploy-image-override/README.md	Remove deploy-image-override example doc.
src/examples/deploy-image-override/deploy.template.yml	Remove deploy-image-override template.
src/configs/worker.yaml	Add built-in default worker.yaml shipped in image.
src/configs/services.yaml	Add built-in default services.yaml shipped in image.
README.md	Update repo messaging and re-point docs to new structure; remove auth/deploy CLI guidance.
Makefile	Update make test mounts/command paths to test/.
lib/worker_config.sh	Replace merged-config approach with “active config path” selection (user mount preferred).
lib/secrets.sh	Adjust internal env skip list; continue env-var secret ref scanning.
lib/runtime_output.sh	Add opt-in runtime JSON output generation.
lib/process_manager.sh	Prefer mounted services config, fall back to built-in; skip supervisor start when no enabled services.
lib/environment.sh	Remove legacy environment coordinator script.
lib/env_handler.sh	Make env file path overridable; add atomic env-file upsert; move configure_environment here; unify env+secrets handling.
lib/cli/service.sh	Add built-in/user services config selection + worker service init.
lib/cli/env.sh	Remove secrets-file status reporting; route env reload through shared configure_environment.
lib/cli/config.sh	Update locations output to show “active config”; route config apply through shared configure_environment.
lib/cli/auth.sh	Remove auth CLI module.
lib/cli.sh	Update help example now that auth is removed.
lib/cleanup.sh	Remove auth cleanup utilities.
lib/auth/gcp.sh	Remove GCP auth implementation.
lib/auth/azure.sh	Remove Azure auth implementation.
lib/auth/aws.sh	Remove AWS auth implementation.
lib/auth.sh	Remove shared auth orchestration implementation.
etc/configs/worker/default.yaml	Remove old built-in config containing actors/creds.
docs/services.md	Update services docs title/links and deployment wording.
docs/secrets.md	Add new secrets boundary/reference doc.
docs/runtime/config.md	Remove legacy runtime config doc (replaced by docs/config.md).
docs/references/README.md	Add references index.
docs/references/container-structure.md	Update related-doc links to new doc locations.
docs/references/cloud-providers-auth.md	Add external-auth options matrix + guidance.
docs/index.md	Remove docs index page (structure reshaped).
docs/development.md	Update development doc links/structure.
docs/deployment.md	Add new deployment doc emphasizing external deployment/auth.
docs/deploy/worker-deployment.md	Remove worker-deployment CLI doc.
docs/deploy/README.md	Remove legacy deploy docs index.
docs/deploy/kubernetes.md	Remove legacy Kubernetes deploy doc.
docs/deploy/image-override.md	Remove legacy image override doc.
docs/core-image.md	Update test-path description to new test/ location.
docs/config.md	Add new worker config doc including runtime output behavior.
docs/cli.md	Update CLI docs (remove auth; add reload/apply guidance).
docs/child-images.md	Update child image guidance (remove generator/deploy.yml references).
docs/authorization.md	Remove legacy authorization doc.
docs/auth/README.md	Remove legacy auth docs index.
docs/auth/gcp.md	Remove legacy GCP auth doc.
docs/auth/azure.md	Remove legacy Azure auth doc.
docs/auth/aws.md	Remove legacy AWS auth doc.
Dockerfile	Copy new built-in configs into image (src/configs/*).
deploy.yml	Remove legacy deploy config example.
deploy-gcp.yml	Remove legacy deploy config example.
ci/prompts/docker-dependency-updater.md	Add Copilot prompt for Docker dependency pin automation.
ci/configs/docker-dependency-probe.yaml	Add probe config for Docker dependency updater workflow.
bin/entrypoint.sh	Use shared config/env/secrets path; add runtime output emission hook.
.rabbit/context.yaml	Add generated repo context metadata.
.github/workflows/docker-ops.yml	Update workflow path filters (test/** + .dockerignore).
.github/workflows/docker-dependency-updater.yml	Add scheduled/dispatch workflow for Docker dependency pin update PRs.
.dockerignore	Reduce build context; include only needed src/configs from src/.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 62 out of 64 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 62 out of 64 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 65 out of 67 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 65 out of 67 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

lib/env_handler.sh:206

• load_environment sources $WORKER_ENV_FILE directly. If that file is ever modified (accidentally or maliciously) to contain non-export shell code, running worker env ... / startup will execute it. Given this file is writable inside the container, it’s safer to validate it contains only export/comments/blank lines before sourcing.