diff --git a/README.md b/README.md index e4c9d55c..b0f5779f 100644 --- a/README.md +++ b/README.md @@ -36,15 +36,15 @@ The Gateway name is configured via the `GATEWAY_API_GATEWAY_NAME` environment va ### Trusted CA bundle (optional) -If the gateway needs to interact with an OIDC issuer (e.g., Keycloak) that uses a self-signed or private CA certificate, create a ConfigMap named `gateway-trusted-ca` in the control plane namespace (default: `hypershell`). The control plane copies this ConfigMap into each tenant namespace and mounts it into gateway pods so they can validate the issuer's TLS certificate when fetching JWKS keys or verifying tokens. +If the gateway needs to interact with an OIDC issuer (e.g., Keycloak) that uses a self-signed or private CA certificate, create a ConfigMap named `gateway-trusted-ca` in the control plane namespace (default: `hypershell-system`). The control plane copies this ConfigMap into each tenant namespace and mounts it into gateway pods so they can validate the issuer's TLS certificate when fetching JWKS keys or verifying tokens. ```shell -kubectl -n hypershell create configmap gateway-trusted-ca --from-file=ca-bundle.crt=/path/to/ca.crt +kubectl -n hypershell-system create configmap gateway-trusted-ca --from-file=ca-bundle.crt=/path/to/ca.crt ``` ### Keycloak OIDC client provisioning (`hypershell-keycloak-admin`) -The control plane provisions an OIDC client in Keycloak for each gateway it reconciles. It authenticates to Keycloak using a confidential client whose credentials are read from a Secret named `hypershell-keycloak-admin` in the control plane namespace (default: `hypershell`). If this Secret is absent at startup, Keycloak integration is silently disabled for the lifetime of that pod. +The control plane provisions an OIDC client in Keycloak for each gateway it reconciles. It authenticates to Keycloak using a confidential client whose credentials are read from a Secret named `hypershell-keycloak-admin` in the control plane namespace (default: `hypershell-system`). If this Secret is absent at startup, Keycloak integration is silently disabled for the lifetime of that pod. #### 1. Create a realm @@ -115,7 +115,7 @@ Retrieve the generated client secret and create the Kubernetes Secret in the con CLIENT_SECRET=$(curl -s "$KEYCLOAK_URL/admin/realms/hypershell/clients/$CLIENT_UUID/client-secret" \ -H "Authorization: Bearer $ADMIN_TOKEN" | jq -r '.value') -kubectl -n hypershell create secret generic hypershell-keycloak-admin \ +kubectl -n hypershell-system create secret generic hypershell-keycloak-admin \ --from-literal=server-url="$KEYCLOAK_URL/" \ --from-literal=realm="hypershell" \ --from-literal=client-id="hypershell-control-plane" \ @@ -125,15 +125,15 @@ kubectl -n hypershell create secret generic hypershell-keycloak-admin \ If you need to rotate the client secret or update any value, delete and recreate the Secret then restart the control plane pod -- the Secret is read once at startup. ```shell -kubectl -n hypershell delete secret hypershell-keycloak-admin +kubectl -n hypershell-system delete secret hypershell-keycloak-admin # recreate with updated values, then: -kubectl -n hypershell rollout restart deployment/hypershell-control-plane +kubectl -n hypershell-system rollout restart deployment/hypershell-controller ``` Confirm the control plane picked up the configuration: ```shell -kubectl -n hypershell logs deployment/hypershell-control-plane | grep -i keycloak +kubectl -n hypershell-system logs deployment/hypershell-controller | grep -i keycloak # Expected: INFO keycloak integration enabled: server=... realm=hypershell ``` @@ -147,26 +147,29 @@ On OpenShift, look up the cluster's default base domain: oc get ingresses.config.openshift.io cluster -o jsonpath='{.spec.domain}' ``` -This typically returns a value like `apps..`. Set this value as `GATEWAY_API_BASE_DOMAIN` on the controller deployment: +This typically returns a value like `apps..`. Set this value as `GATEWAY_API_BASE_DOMAIN` on the controller deployment using one of: ```shell -oc set env deployment/hypershell-controller -n hypershell \ +# Option 1: Patch the deployment directly +oc set env deployment/hypershell-controller -n hypershell-system \ GATEWAY_API_BASE_DOMAIN="$(oc get ingresses.config.openshift.io cluster -o jsonpath='{.spec.domain}')" ``` -Or edit `components/api-server/deploy/openshift/controller.yaml` and replace the placeholder value before applying. +Or apply a kustomize patch via `deploy/openshift/kustomization.yaml` with your specific base domain value (see that file for the patch structure). ### Control plane environment variables -| Variable | Default | Description | -|---|---|---| -| `HYPERSHELL_GRPC_SERVER_ADDR` | `localhost:9000` | gRPC address of the API server | -| `HYPERSHELL_API_SERVER_URL` | `http://localhost:8000` | HTTP address of the API server | -| `HYPERSHELL_NAMESPACE` | `hypershell` | Namespace the control plane runs in (used for trusted CA bundle source) | -| `GATEWAY_API_GATEWAY_NAME` | *(required)* | Name of the pre-existing Gateway resource that tenant GRPCRoutes attach to | -| `GATEWAY_API_GATEWAY_NAMESPACE` | `openshift-ingress` | Namespace where the pre-existing Gateway resource lives | -| `GATEWAY_API_BASE_DOMAIN` | *(none)* | Base domain for tenant hostname generation (e.g., `openshell.example.com` → `gw-.openshell.example.com`) | -| `GATEWAY_MANIFESTS_DIR` | `/manifests/gateway` | Path to gateway manifest templates | +| Variable | Default | Required | Description | +|---|---|---|---| +| `HYPERSHELL_GRPC_SERVER_ADDR` | `localhost:9000` | ✓ | gRPC address of the API server | +| `HYPERSHELL_API_SERVER_URL` | `http://localhost:8000` | ✓ | HTTP address of the API server | +| `HYPERSHELL_NAMESPACE` | `hypershell-system` | ✓ | Namespace the control plane runs in (used for trusted CA bundle source) | +| `GATEWAY_IMAGE` | *(none)* | **✓ required** | Container image for tenant gateways (pinned by digest; no fallback). Set in `deploy/base/controller.yaml` | +| `GATEWAY_SUPERVISOR_IMAGE` | *(none)* | **✓ required** | Container image for gateway supervisors (pinned by digest; no fallback). Set in `deploy/base/controller.yaml` | +| `GATEWAY_API_GATEWAY_NAME` | *(required)* | ✓ | Name of the pre-existing Gateway resource that tenant GRPCRoutes attach to | +| `GATEWAY_API_GATEWAY_NAMESPACE` | `openshift-ingress` | ✓ | Namespace where the pre-existing Gateway resource lives | +| `GATEWAY_API_BASE_DOMAIN` | *(none)* | | Base domain for tenant hostname generation (e.g., `openshell.example.com` → `gw-.openshell.example.com`) | +| `GATEWAY_MANIFESTS_DIR` | `/manifests/gateway` | ✓ | Path to gateway manifest templates | ## Observability diff --git a/components/api-server/Makefile b/components/api-server/Makefile index 9e828d87..c314ef4e 100644 --- a/components/api-server/Makefile +++ b/components/api-server/Makefile @@ -125,76 +125,3 @@ proto: .PHONY: proto-clean proto-clean: rm -rf pkg/api/grpc/ - -KIND_CLUSTER_NAME?=hypershell-dev -KIND_API_PORT?=23080 -IMAGE_NAME=hypershell -CONTROLLER_IMAGE_NAME=hypershell-controller -IMAGE_TAG=dev -REPO_ROOT=$(shell git rev-parse --show-toplevel) - -.PHONY: image -image: - $(CONTAINER_ENGINE) build -t $(IMAGE_NAME):$(IMAGE_TAG) \ - -f Dockerfile \ - --build-arg GIT_VERSION=$(build_version) \ - --build-arg BUILD_TIME="$(build_time)" \ - . - -.PHONY: image-controller -image-controller: - $(CONTAINER_ENGINE) build -t $(CONTROLLER_IMAGE_NAME):$(IMAGE_TAG) \ - -f $(REPO_ROOT)/components/control-plane/Dockerfile \ - $(REPO_ROOT) - -.PHONY: images -images: image image-controller - -.PHONY: kind-up -kind-up: images - kind create cluster --name $(KIND_CLUSTER_NAME) --config deploy/kind/kind-config.yaml || true - rm -f /tmp/$(IMAGE_NAME)-$(IMAGE_TAG).tar /tmp/$(CONTROLLER_IMAGE_NAME)-$(IMAGE_TAG).tar - $(CONTAINER_ENGINE) save -o /tmp/$(IMAGE_NAME)-$(IMAGE_TAG).tar $(IMAGE_NAME):$(IMAGE_TAG) - kind load image-archive /tmp/$(IMAGE_NAME)-$(IMAGE_TAG).tar --name $(KIND_CLUSTER_NAME) - $(CONTAINER_ENGINE) save -o /tmp/$(CONTROLLER_IMAGE_NAME)-$(IMAGE_TAG).tar $(CONTROLLER_IMAGE_NAME):$(IMAGE_TAG) - kind load image-archive /tmp/$(CONTROLLER_IMAGE_NAME)-$(IMAGE_TAG).tar --name $(KIND_CLUSTER_NAME) - kubectl kustomize deploy/kind/ | kubectl apply -f - - @echo "Waiting for PostgreSQL..." - kubectl wait --for=condition=ready pod -l app=hypershell-postgres -n hypershell --timeout=120s - @echo "Waiting for API server..." - kubectl wait --for=condition=available deployment/hypershell-api-server -n hypershell --timeout=120s - @echo "Waiting for controller..." - kubectl wait --for=condition=available deployment/hypershell-controller -n hypershell --timeout=120s - @echo "" - @echo "HyperShell is running!" - @echo " API: http://localhost:$(KIND_API_PORT)/api/hypershell/v1/fleets" - @echo " API Server Logs: kubectl logs -f -l app=hypershell-api-server -n hypershell" - @echo " Controller Logs: kubectl logs -f -l app=hypershell-controller -n hypershell" - -.PHONY: kind-down -kind-down: - kind delete cluster --name $(KIND_CLUSTER_NAME) - -.PHONY: kind-rebuild -kind-rebuild: images - rm -f /tmp/$(IMAGE_NAME)-$(IMAGE_TAG).tar /tmp/$(CONTROLLER_IMAGE_NAME)-$(IMAGE_TAG).tar - $(CONTAINER_ENGINE) save -o /tmp/$(IMAGE_NAME)-$(IMAGE_TAG).tar $(IMAGE_NAME):$(IMAGE_TAG) - kind load image-archive /tmp/$(IMAGE_NAME)-$(IMAGE_TAG).tar --name $(KIND_CLUSTER_NAME) - $(CONTAINER_ENGINE) save -o /tmp/$(CONTROLLER_IMAGE_NAME)-$(IMAGE_TAG).tar $(CONTROLLER_IMAGE_NAME):$(IMAGE_TAG) - kind load image-archive /tmp/$(CONTROLLER_IMAGE_NAME)-$(IMAGE_TAG).tar --name $(KIND_CLUSTER_NAME) - kubectl kustomize deploy/kind/ | kubectl apply -f - - kubectl rollout restart deployment/hypershell-api-server -n hypershell - kubectl rollout restart deployment/hypershell-controller -n hypershell - kubectl wait --for=condition=available deployment/hypershell-api-server -n hypershell --timeout=120s - kubectl wait --for=condition=available deployment/hypershell-controller -n hypershell --timeout=120s - -.PHONY: kind-status -kind-status: - @echo "=== Cluster ===" - kubectl cluster-info --context kind-$(KIND_CLUSTER_NAME) 2>/dev/null || echo "Cluster not running" - @echo "" - @echo "=== Pods ===" - kubectl get pods -n hypershell 2>/dev/null || echo "Namespace not found" - @echo "" - @echo "=== Services ===" - kubectl get svc -n hypershell 2>/dev/null || echo "Namespace not found" diff --git a/components/api-server/deploy/ibm/controller-clusterrbac.yaml b/components/api-server/deploy/ibm/controller-clusterrbac.yaml deleted file mode 100644 index 3a797108..00000000 --- a/components/api-server/deploy/ibm/controller-clusterrbac.yaml +++ /dev/null @@ -1,90 +0,0 @@ -# Cluster-scoped controller RBAC for full tenant reconciliation. -# -# The self-contained ../openshift tree ships only a narrow Role (gateways in -# openshift-ingress), which is insufficient once the control plane reconciles -# whole tenant gateways: it creates per-tenant namespaces, Deployments, -# Services, Secrets, NetworkPolicies, and -- in Route ingress mode -- OpenShift -# Routes, all cluster-wide. This mirrors deploy/base/controller-rbac.yaml (the -# GitOps tree's canonical grant), bound to the hypershell/hypershell-controller -# service account used by ../openshift. -# -# Not IBM-specific in itself; carried in this overlay until the ../openshift base -# grows the same ClusterRole. See specs/platform/global-architecture.spec.md. -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: hypershell-controller - labels: - app.kubernetes.io/name: hypershell - app.kubernetes.io/component: controller -rules: - - apiGroups: [""] - resources: ["namespaces", "secrets", "configmaps", "services", "serviceaccounts", "persistentvolumeclaims"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - - apiGroups: ["apps"] - resources: ["deployments", "statefulsets"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - - apiGroups: ["batch"] - resources: ["jobs"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - - apiGroups: ["rbac.authorization.k8s.io"] - resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - - apiGroups: ["networking.k8s.io"] - resources: ["networkpolicies"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - - apiGroups: ["cert-manager.io"] - resources: ["issuers", "certificates"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - - apiGroups: ["gateway.networking.k8s.io"] - resources: ["gateways", "grpcroutes", "backendtlspolicies"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - - apiGroups: ["route.openshift.io"] - resources: ["routes"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - # Route ingress mode derives an explicit spec.host (gw-.). - # OpenShift gates setting a Route's host behind the routes/custom-host - # subresource; without this the Route is rejected - # ("you do not have permission to set the host field of the route"). - - apiGroups: ["route.openshift.io"] - resources: ["routes/custom-host"] - verbs: ["create", "update"] - # The controller binds the sandbox service account to the privileged SCC - # (roleRef system:openshift:scc:privileged) so agent sandboxes can run. RBAC - # escalation prevention requires the controller to itself hold "use" of that - # SCC before it can grant it; without this the RoleBinding create is forbidden. - - apiGroups: ["security.openshift.io"] - resources: ["securitycontextconstraints"] - resourceNames: ["privileged"] - verbs: ["use"] - - apiGroups: ["apiextensions.k8s.io"] - resources: ["customresourcedefinitions"] - verbs: ["get", "list"] - - apiGroups: [""] - resources: ["nodes"] - verbs: ["get", "list", "watch"] - - apiGroups: [""] - resources: ["pods", "events"] - verbs: ["get", "list", "watch"] - - apiGroups: ["authentication.k8s.io"] - resources: ["tokenreviews"] - verbs: ["create"] - - apiGroups: ["agents.x-k8s.io"] - resources: ["sandboxes", "sandboxes/status"] - verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: hypershell-controller - labels: - app.kubernetes.io/name: hypershell - app.kubernetes.io/component: controller -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: hypershell-controller -subjects: - - kind: ServiceAccount - name: hypershell-controller - namespace: hypershell diff --git a/components/api-server/deploy/ibm/kustomization.yaml b/components/api-server/deploy/ibm/kustomization.yaml deleted file mode 100644 index 2a5a6b80..00000000 --- a/components/api-server/deploy/ibm/kustomization.yaml +++ /dev/null @@ -1,87 +0,0 @@ -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization - -# IBM Cloud (ROKS) overlay for the dev/no-auth platform deploy. -# -# This is the environment adapter for the self-contained OpenShift deploy in -# ../openshift (namespace `hypershell`, the tree documented by the -# `deploy-cluster` skill). It changes only what the ROKS environment forces us -# to change, leaving the architecture identical: -# -# 1. Ingress mode -> Route. ROKS is HyperShift-hosted and cannot run the -# CIO-managed Gateway API (OSSM images unpullable, node mirroring / IDMS -# owned by the HostedCluster). The control plane instead emits passthrough -# OpenShift Routes on IBM's free "*.containers.appdomain.cloud" wildcard, -# selected by GATEWAY_INGRESS_MODE=route. Passthrough preserves the gateway -# pod's per-tenant self-signed CA TLS end to end -- no shared Gateway, -# wildcard cert, cert-manager ClusterIssuer, or Route53. -# -# 2. Image sources -> internal registry. ROKS worker nodes cannot reach -# quay.io or registry.access.redhat.com (egress is restricted to IBM -# registries), so all images are mirrored into the cluster internal -# registry and referenced by their in-cluster service address -# (image-registry.openshift-image-registry.svc:5000/hypershell/...), which -# the kubelet can resolve and pull. -# -# See: -# specs/platform/global-architecture.spec.md (§ IBM Cloud Cloud Hub - Route ingress mode) -# skills/deploy/ibm-cluster/SKILL.md (Step 5) -# skills/deploy/deploy-cluster/SKILL.md (Cloud-Hub Parameter Overrides) - -resources: - - ../openshift - - controller-clusterrbac.yaml - -# --- point every image at the mirrored copy in the internal registry --- -# The kubelet pulls these by their internal service address; the default -# service account in the `hypershell` namespace already has system:image-puller -# for same-namespace image streams. -images: - - name: quay.io/redhat-services-prod/hcm-eng-prod-tenant/hypershell-main/hypershell-api-server-main - newName: image-registry.openshift-image-registry.svc:5000/hypershell/hypershell-api-server - newTag: dev - - name: quay.io/redhat-services-prod/hcm-eng-prod-tenant/hypershell-main/hypershell-control-plane-main - newName: image-registry.openshift-image-registry.svc:5000/hypershell/hypershell-controller - newTag: dev - - name: registry.access.redhat.com/hi/postgresql - newName: image-registry.openshift-image-registry.svc:5000/hypershell/postgresql - newTag: "18.4" - -patches: - # --- select Route ingress mode + this cluster's ingress subdomain --- - # Strategic-merge on the env list (merged by name): adds GATEWAY_INGRESS_MODE - # and overrides the base-domain placeholder with the ROKS ingress subdomain, so - # tenant gateways get "gw-." hosts under the free wildcard. - - patch: | - apiVersion: apps/v1 - kind: Deployment - metadata: - name: hypershell-controller - namespace: hypershell - spec: - template: - spec: - containers: - - name: controller - env: - - name: GATEWAY_INGRESS_MODE - value: route - - name: GATEWAY_API_BASE_DOMAIN - value: hysh-ibm-01-4c28435107377e996c6eb39230b7bcf5-0000.us-east.containers.appdomain.cloud - # ROKS nodes cannot pull Docker Hub, so the per-tenant gateway - # database image is served from the internal registry (mirror - # of docker.io/library/postgres:18 in the openshift namespace, - # globally pullable). The name stays "postgres" (not RHEL - # "postgresql-"), so the reconciler's variant detection - # (reconciler.go:1186) keeps the POSTGRES_* env + data-path - # conventions that match this image. - - name: HYPERSHELL_DATABASE_IMAGE - value: image-registry.openshift-image-registry.svc:5000/openshift/postgres:18 - # Tenant sandbox pods launch from this base image. ROKS nodes - # cannot pull ghcr.io, so it is served from the internal - # registry (mirror of - # ghcr.io/nvidia/openshell-community/sandboxes/base:latest in - # the openshift namespace, globally pullable). Without this the - # Sandbox CR is admitted but its pod ImagePullBackOffs. - - name: GATEWAY_SANDBOX_IMAGE - value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-sandbox-base:latest diff --git a/components/api-server/deploy/kind/api-server-nodeport.yaml b/components/api-server/deploy/kind/api-server-nodeport.yaml deleted file mode 100644 index a294a721..00000000 --- a/components/api-server/deploy/kind/api-server-nodeport.yaml +++ /dev/null @@ -1,14 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: hypershell-api-server-nodeport - namespace: hypershell -spec: - type: NodePort - selector: - app: hypershell-api-server - ports: - - name: http - port: 8000 - targetPort: 8000 - nodePort: 30080 diff --git a/components/api-server/deploy/kind/api-server.yaml b/components/api-server/deploy/kind/api-server.yaml deleted file mode 100644 index 1506771a..00000000 --- a/components/api-server/deploy/kind/api-server.yaml +++ /dev/null @@ -1,100 +0,0 @@ -apiVersion: v1 -kind: ServiceAccount -metadata: - name: hypershell-api-server - namespace: hypershell ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: hypershell-api-server - namespace: hypershell -spec: - replicas: 1 - selector: - matchLabels: - app: hypershell-api-server - template: - metadata: - labels: - app: hypershell-api-server - spec: - serviceAccountName: hypershell-api-server - initContainers: - - name: migrate - image: localhost/hypershell:dev - imagePullPolicy: Never - command: - - /usr/local/bin/hypershell - - migrate - - --db-host-file=/secrets/db.host - - --db-port-file=/secrets/db.port - - --db-name-file=/secrets/db.name - - --db-user-file=/secrets/db.user - - --db-password-file=/secrets/db.password - - --db-sslmode=disable - volumeMounts: - - name: db-secrets - mountPath: /secrets - readOnly: true - containers: - - name: api-server - image: localhost/hypershell:dev - imagePullPolicy: Never - command: - - /usr/local/bin/hypershell - - serve - - --api-server-bindaddress=0.0.0.0:8000 - - --grpc-server-bindaddress=0.0.0.0:9000 - - --health-check-server-bindaddress=0.0.0.0:4434 - - --enable-authz=false - - --enable-jwt=false - - --db-host-file=/secrets/db.host - - --db-port-file=/secrets/db.port - - --db-name-file=/secrets/db.name - - --db-user-file=/secrets/db.user - - --db-password-file=/secrets/db.password - - --db-sslmode=disable - ports: - - containerPort: 8000 - name: http - - containerPort: 9000 - name: grpc - - containerPort: 4434 - name: health - livenessProbe: - httpGet: - path: /healthcheck - port: 4434 - initialDelaySeconds: 10 - periodSeconds: 10 - readinessProbe: - httpGet: - path: /healthcheck - port: 4434 - initialDelaySeconds: 5 - periodSeconds: 5 - volumeMounts: - - name: db-secrets - mountPath: /secrets - readOnly: true - volumes: - - name: db-secrets - secret: - secretName: hypershell-db ---- -apiVersion: v1 -kind: Service -metadata: - name: hypershell-api-server - namespace: hypershell -spec: - selector: - app: hypershell-api-server - ports: - - name: http - port: 8000 - targetPort: 8000 - - name: grpc - port: 9000 - targetPort: 9000 diff --git a/components/api-server/deploy/kind/controller-gateway-rbac.yaml b/components/api-server/deploy/kind/controller-gateway-rbac.yaml deleted file mode 100644 index 183bc2fe..00000000 --- a/components/api-server/deploy/kind/controller-gateway-rbac.yaml +++ /dev/null @@ -1,27 +0,0 @@ -# Must match GATEWAY_API_GATEWAY_NAMESPACE (default: openshift-ingress) -apiVersion: rbac.authorization.k8s.io/v1 -kind: Role -metadata: - name: hypershell-gateway-manager - namespace: openshift-ingress -rules: - - apiGroups: ["gateway.networking.k8s.io"] - resources: ["gateways"] - verbs: ["get", "list", "create", "update", "patch", "delete"] - - apiGroups: [""] - resources: ["secrets"] - verbs: ["get", "list"] ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: RoleBinding -metadata: - name: hypershell-gateway-manager - namespace: openshift-ingress -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: Role - name: hypershell-gateway-manager -subjects: - - kind: ServiceAccount - name: hypershell-controller - namespace: hypershell diff --git a/components/api-server/deploy/kind/controller.yaml b/components/api-server/deploy/kind/controller.yaml deleted file mode 100644 index e87fd7b0..00000000 --- a/components/api-server/deploy/kind/controller.yaml +++ /dev/null @@ -1,35 +0,0 @@ -apiVersion: v1 -kind: ServiceAccount -metadata: - name: hypershell-controller - namespace: hypershell ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: hypershell-controller - namespace: hypershell -spec: - replicas: 1 - selector: - matchLabels: - app: hypershell-controller - template: - metadata: - labels: - app: hypershell-controller - spec: - serviceAccountName: hypershell-controller - containers: - - name: controller - image: localhost/hypershell-controller:dev - imagePullPolicy: Never - env: - - name: HYPERSHELL_GRPC_SERVER_ADDR - value: "hypershell-api-server:9000" - - name: HYPERSHELL_API_SERVER_URL - value: "http://hypershell-api-server:8000" - - name: HYPERSHELL_NAMESPACE - value: "hypershell" - - name: HYPERSHELL_LOG_LEVEL - value: "info" diff --git a/components/api-server/deploy/kind/kind-config.yaml b/components/api-server/deploy/kind/kind-config.yaml deleted file mode 100644 index 37cad612..00000000 --- a/components/api-server/deploy/kind/kind-config.yaml +++ /dev/null @@ -1,8 +0,0 @@ -kind: Cluster -apiVersion: kind.x-k8s.io/v1alpha4 -nodes: - - role: control-plane - extraPortMappings: - - containerPort: 30080 - hostPort: 23080 - protocol: TCP diff --git a/components/api-server/deploy/kind/kustomization.yaml b/components/api-server/deploy/kind/kustomization.yaml deleted file mode 100644 index 7116adf3..00000000 --- a/components/api-server/deploy/kind/kustomization.yaml +++ /dev/null @@ -1,10 +0,0 @@ -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization - -resources: - - namespace.yaml - - postgres.yaml - - api-server.yaml - - api-server-nodeport.yaml - - controller.yaml - - controller-gateway-rbac.yaml diff --git a/components/api-server/deploy/kind/namespace.yaml b/components/api-server/deploy/kind/namespace.yaml deleted file mode 100644 index bfc59bac..00000000 --- a/components/api-server/deploy/kind/namespace.yaml +++ /dev/null @@ -1,4 +0,0 @@ -apiVersion: v1 -kind: Namespace -metadata: - name: hypershell diff --git a/components/api-server/deploy/kind/postgres.yaml b/components/api-server/deploy/kind/postgres.yaml deleted file mode 100644 index 0b015ea3..00000000 --- a/components/api-server/deploy/kind/postgres.yaml +++ /dev/null @@ -1,79 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: - name: hypershell-db - namespace: hypershell -type: Opaque -stringData: - db.host: hypershell-postgres - db.port: "5432" - db.name: hypershell - db.user: postgres - db.password: postgres ---- -apiVersion: v1 -kind: Service -metadata: - name: hypershell-postgres - namespace: hypershell -spec: - selector: - app: hypershell-postgres - ports: - - port: 5432 - targetPort: 5432 ---- -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: hypershell-postgres - namespace: hypershell -spec: - serviceName: hypershell-postgres - replicas: 1 - selector: - matchLabels: - app: hypershell-postgres - template: - metadata: - labels: - app: hypershell-postgres - spec: - securityContext: - runAsNonRoot: true - runAsUser: 26 - fsGroup: 26 - containers: - - name: postgres - image: registry.access.redhat.com/hi/postgresql:18.4@sha256:9b1917bf15a3b3a6a99b94ab75db1bfde3f434990e881c69d527417d2c035a09 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - ports: - - containerPort: 5432 - env: - - name: POSTGRES_USER - value: postgres - - name: POSTGRES_PASSWORD - value: postgres - - name: POSTGRES_DB - value: hypershell - - name: PGDATA - value: /var/lib/postgresql/data/pgdata - readinessProbe: - exec: - command: ["pg_isready", "-U", "postgres"] - initialDelaySeconds: 5 - periodSeconds: 5 - volumeMounts: - - name: data - mountPath: /var/lib/postgresql/data - volumeClaimTemplates: - - metadata: - name: data - spec: - accessModes: ["ReadWriteOnce"] - resources: - requests: - storage: 1Gi diff --git a/components/api-server/deploy/openshift/api-server.yaml b/components/api-server/deploy/openshift/api-server.yaml deleted file mode 100644 index d5c7b654..00000000 --- a/components/api-server/deploy/openshift/api-server.yaml +++ /dev/null @@ -1,110 +0,0 @@ -apiVersion: v1 -kind: ServiceAccount -metadata: - name: hypershell-api-server - namespace: hypershell ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: hypershell-api-server - namespace: hypershell -spec: - replicas: 1 - selector: - matchLabels: - app: hypershell-api-server - template: - metadata: - labels: - app: hypershell-api-server - spec: - serviceAccountName: hypershell-api-server - securityContext: - runAsNonRoot: true - initContainers: - - name: migrate - image: quay.io/redhat-services-prod/hcm-eng-prod-tenant/hypershell-main/hypershell-api-server-main@sha256:4b2898091ffb72ff3f7f60e6cc41bf2219065ccad05c901dffd4b4baa9774252 - imagePullPolicy: Always - command: - - /usr/local/bin/hypershell - - migrate - - --db-host-file=/secrets/db.host - - --db-port-file=/secrets/db.port - - --db-name-file=/secrets/db.name - - --db-user-file=/secrets/db.user - - --db-password-file=/secrets/db.password - - --db-sslmode=disable - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - name: db-secrets - mountPath: /secrets - readOnly: true - containers: - - name: api-server - image: quay.io/redhat-services-prod/hcm-eng-prod-tenant/hypershell-main/hypershell-api-server-main@sha256:4b2898091ffb72ff3f7f60e6cc41bf2219065ccad05c901dffd4b4baa9774252 - imagePullPolicy: Always - command: - - /usr/local/bin/hypershell - - serve - - --api-server-bindaddress=0.0.0.0:8000 - - --grpc-server-bindaddress=0.0.0.0:9000 - - --health-check-server-bindaddress=0.0.0.0:4434 - - --enable-authz=false - - --enable-jwt=false - - --db-host-file=/secrets/db.host - - --db-port-file=/secrets/db.port - - --db-name-file=/secrets/db.name - - --db-user-file=/secrets/db.user - - --db-password-file=/secrets/db.password - - --db-sslmode=disable - ports: - - containerPort: 8000 - name: http - - containerPort: 9000 - name: grpc - - containerPort: 4434 - name: health - livenessProbe: - httpGet: - path: /healthcheck - port: 4434 - initialDelaySeconds: 10 - periodSeconds: 10 - readinessProbe: - httpGet: - path: /healthcheck - port: 4434 - initialDelaySeconds: 5 - periodSeconds: 5 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - name: db-secrets - mountPath: /secrets - readOnly: true - volumes: - - name: db-secrets - secret: - secretName: hypershell-db ---- -apiVersion: v1 -kind: Service -metadata: - name: hypershell-api-server - namespace: hypershell -spec: - selector: - app: hypershell-api-server - ports: - - name: http - port: 8000 - targetPort: 8000 - - name: grpc - port: 9000 - targetPort: 9000 diff --git a/components/api-server/deploy/openshift/controller-gateway-rbac.yaml b/components/api-server/deploy/openshift/controller-gateway-rbac.yaml deleted file mode 100644 index 183bc2fe..00000000 --- a/components/api-server/deploy/openshift/controller-gateway-rbac.yaml +++ /dev/null @@ -1,27 +0,0 @@ -# Must match GATEWAY_API_GATEWAY_NAMESPACE (default: openshift-ingress) -apiVersion: rbac.authorization.k8s.io/v1 -kind: Role -metadata: - name: hypershell-gateway-manager - namespace: openshift-ingress -rules: - - apiGroups: ["gateway.networking.k8s.io"] - resources: ["gateways"] - verbs: ["get", "list", "create", "update", "patch", "delete"] - - apiGroups: [""] - resources: ["secrets"] - verbs: ["get", "list"] ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: RoleBinding -metadata: - name: hypershell-gateway-manager - namespace: openshift-ingress -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: Role - name: hypershell-gateway-manager -subjects: - - kind: ServiceAccount - name: hypershell-controller - namespace: hypershell diff --git a/components/api-server/deploy/openshift/controller.yaml b/components/api-server/deploy/openshift/controller.yaml deleted file mode 100644 index 07fa1a41..00000000 --- a/components/api-server/deploy/openshift/controller.yaml +++ /dev/null @@ -1,43 +0,0 @@ -apiVersion: v1 -kind: ServiceAccount -metadata: - name: hypershell-controller - namespace: hypershell ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: hypershell-controller - namespace: hypershell -spec: - replicas: 1 - selector: - matchLabels: - app: hypershell-controller - template: - metadata: - labels: - app: hypershell-controller - spec: - serviceAccountName: hypershell-controller - securityContext: - runAsNonRoot: true - containers: - - name: controller - image: quay.io/redhat-services-prod/hcm-eng-prod-tenant/hypershell-main/hypershell-control-plane-main@sha256:fb09ac10910766e6072a6c0fe2b0d9121898ef19001a07ab85303802ccef1705 - imagePullPolicy: Always - env: - - name: HYPERSHELL_GRPC_SERVER_ADDR - value: "hypershell-api-server:9000" - - name: HYPERSHELL_API_SERVER_URL - value: "http://hypershell-api-server:8000" - - name: HYPERSHELL_NAMESPACE - value: "hypershell" - - name: HYPERSHELL_LOG_LEVEL - value: "info" - - name: GATEWAY_API_BASE_DOMAIN - value: "REPLACE_WITH_CLUSTER_BASE_DOMAIN" - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] diff --git a/components/api-server/deploy/openshift/kustomization.yaml b/components/api-server/deploy/openshift/kustomization.yaml deleted file mode 100644 index 9dedec7b..00000000 --- a/components/api-server/deploy/openshift/kustomization.yaml +++ /dev/null @@ -1,10 +0,0 @@ -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization - -resources: - - namespace.yaml - - postgres.yaml - - api-server.yaml - - controller.yaml - - controller-gateway-rbac.yaml - - route.yaml diff --git a/components/api-server/deploy/openshift/namespace.yaml b/components/api-server/deploy/openshift/namespace.yaml deleted file mode 100644 index bfc59bac..00000000 --- a/components/api-server/deploy/openshift/namespace.yaml +++ /dev/null @@ -1,4 +0,0 @@ -apiVersion: v1 -kind: Namespace -metadata: - name: hypershell diff --git a/components/api-server/deploy/openshift/postgres.yaml b/components/api-server/deploy/openshift/postgres.yaml deleted file mode 100644 index a988f7fb..00000000 --- a/components/api-server/deploy/openshift/postgres.yaml +++ /dev/null @@ -1,77 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: - name: hypershell-db - namespace: hypershell -type: Opaque -stringData: - db.host: hypershell-postgres - db.port: "5432" - db.name: hypershell - db.user: hypershell - db.password: hypershell ---- -apiVersion: v1 -kind: Service -metadata: - name: hypershell-postgres - namespace: hypershell -spec: - selector: - app: hypershell-postgres - ports: - - port: 5432 - targetPort: 5432 ---- -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: hypershell-postgres - namespace: hypershell -spec: - serviceName: hypershell-postgres - replicas: 1 - selector: - matchLabels: - app: hypershell-postgres - template: - metadata: - labels: - app: hypershell-postgres - spec: - securityContext: - runAsNonRoot: true - containers: - - name: postgres - image: registry.access.redhat.com/hi/postgresql:18.4@sha256:9b1917bf15a3b3a6a99b94ab75db1bfde3f434990e881c69d527417d2c035a09 - ports: - - containerPort: 5432 - env: - - name: POSTGRES_USER - value: hypershell - - name: POSTGRES_PASSWORD - value: hypershell - - name: POSTGRES_DB - value: hypershell - - name: PGDATA - value: /var/lib/postgresql/data/pgdata - readinessProbe: - exec: - command: ["pg_isready", "-U", "hypershell"] - initialDelaySeconds: 10 - periodSeconds: 5 - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - volumeMounts: - - name: data - mountPath: /var/lib/postgresql/data - volumeClaimTemplates: - - metadata: - name: data - spec: - accessModes: ["ReadWriteOnce"] - resources: - requests: - storage: 1Gi diff --git a/components/api-server/deploy/openshift/route.yaml b/components/api-server/deploy/openshift/route.yaml deleted file mode 100644 index 6219a328..00000000 --- a/components/api-server/deploy/openshift/route.yaml +++ /dev/null @@ -1,14 +0,0 @@ -apiVersion: route.openshift.io/v1 -kind: Route -metadata: - name: hypershell-api - namespace: hypershell -spec: - to: - kind: Service - name: hypershell-api-server - port: - targetPort: http - tls: - termination: edge - insecureEdgeTerminationPolicy: Redirect diff --git a/components/control-plane/internal/gateway/config.go b/components/control-plane/internal/gateway/config.go index 06dee8f8..09b66055 100644 --- a/components/control-plane/internal/gateway/config.go +++ b/components/control-plane/internal/gateway/config.go @@ -37,28 +37,18 @@ const defaultOAuth2ProxyImage = "quay.io/oauth2-proxy/oauth2-proxy:v7.7.1" type StaticImageDefaults struct{} -const defaultGatewayImage = "ghcr.io/nvidia/openshell/gateway:0.0.109" -const defaultSupervisorImage = "ghcr.io/nvidia/openshell/supervisor:0.0.109" - // DefaultGatewayImage resolves the gateway server (and certgen) image used when -// a Gateway resource does not specify one. Overridable via GATEWAY_IMAGE so -// clusters whose nodes cannot reach ghcr.io (e.g. IBM ROKS) can point it at an -// in-cluster registry mirror, mirroring the GATEWAY_SANDBOX_IMAGE override. +// a Gateway resource does not specify one. Must be set via GATEWAY_IMAGE environment +// variable; reconciliation will fail if not provided. func (StaticImageDefaults) DefaultGatewayImage() string { - if v := os.Getenv("GATEWAY_IMAGE"); v != "" { - return v - } - return defaultGatewayImage + return os.Getenv("GATEWAY_IMAGE") } // DefaultSupervisorImage resolves the supervisor sidecar image used when a -// Gateway resource does not specify one. Overridable via GATEWAY_SUPERVISOR_IMAGE -// for the same ghcr.io-unreachable clusters as DefaultGatewayImage. +// Gateway resource does not specify one. Must be set via GATEWAY_SUPERVISOR_IMAGE environment +// variable; reconciliation will fail if not provided. func (StaticImageDefaults) DefaultSupervisorImage() string { - if v := os.Getenv("GATEWAY_SUPERVISOR_IMAGE"); v != "" { - return v - } - return defaultSupervisorImage + return os.Getenv("GATEWAY_SUPERVISOR_IMAGE") } func (StaticImageDefaults) DefaultDatabaseImage() string { diff --git a/components/control-plane/internal/gateway/manifests.go b/components/control-plane/internal/gateway/manifests.go index 2cef601e..2a9cd26c 100644 --- a/components/control-plane/internal/gateway/manifests.go +++ b/components/control-plane/internal/gateway/manifests.go @@ -1,6 +1,8 @@ package gateway import ( + "cmp" + "errors" "fmt" "os" "path/filepath" @@ -86,20 +88,22 @@ func ApplyManifestToNamespace(manifest *unstructured.Unstructured, namespace str manifestJSON := string(jsonBytes) manifestJSON = strings.ReplaceAll(manifestJSON, "NAMESPACE_PLACEHOLDER", namespace) - supervisorImage := images.DefaultSupervisorImage() - if config.SupervisorImage != "" { - supervisorImage = config.SupervisorImage + supervisorImage := cmp.Or(config.SupervisorImage, images.DefaultSupervisorImage()) + if supervisorImage == "" { + return nil, errors.New("supervisor image is not configured and no default is available") } + manifestJSON = strings.ReplaceAll(manifestJSON, "SUPERVISOR_IMAGE_PLACEHOLDER", supervisorImage) // Replace SANDBOX_IMAGE_PLACEHOLDER before IMAGE_PLACEHOLDER because the // shorter string is a substring of the longer one. manifestJSON = strings.ReplaceAll(manifestJSON, "SANDBOX_IMAGE_PLACEHOLDER", images.DefaultSandboxImage()) - image := images.DefaultGatewayImage() - if config.Image != "" { - image = config.Image + image := cmp.Or(config.Image, images.DefaultGatewayImage()) + if image == "" { + return nil, errors.New("gateway image is not configured and no default is available") } + manifestJSON = strings.ReplaceAll(manifestJSON, "IMAGE_PLACEHOLDER", image) result := &unstructured.Unstructured{} diff --git a/components/control-plane/internal/gateway/validation_test.go b/components/control-plane/internal/gateway/validation_test.go index 06fcef95..9a668b19 100644 --- a/components/control-plane/internal/gateway/validation_test.go +++ b/components/control-plane/internal/gateway/validation_test.go @@ -14,6 +14,7 @@ func TestValidateImageReference(t *testing.T) { {name: "bare name with tag", ref: "postgres:18", wantErr: false}, {name: "docker hub library path", ref: "docker.io/library/postgres:18", wantErr: false}, {name: "ghcr multi-segment path with tag", ref: "ghcr.io/nvidia/openshell/gateway:0.0.101", wantErr: false}, + {name: "multi-segment path with tag and reference", ref: "quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd", wantErr: false}, {name: "quay long path with tag", ref: "quay.io/redhat-services-prod/hcm-eng-prod-tenant/hypershell-api-server-main:dev", wantErr: false}, {name: "digest reference", ref: "registry.redhat.io/rhel9/postgresql-16@sha256:" + "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", wantErr: false}, // In-cluster registry service address carries an explicit port; this is diff --git a/deploy/base/controller.yaml b/deploy/base/controller.yaml index 307322c7..efea84f0 100644 --- a/deploy/base/controller.yaml +++ b/deploy/base/controller.yaml @@ -46,6 +46,10 @@ spec: # sibling control-plane -> api-server watch channel (no mTLS). - name: HYPERSHELL_SERVICE_ACCOUNT_PROVISIONER_BIND_ADDRESS value: "0.0.0.0:9443" + - name: GATEWAY_IMAGE + value: quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd + - name: GATEWAY_SUPERVISOR_IMAGE + value: quay.io/opendatahub/odh-openshell-supervisor:v0.0.109-rhaiv.0@sha256:96e21135c18bc9f6f4d1dfd0cccae3c91769ef4d87da2e470eca4b56a24b2152 ports: - containerPort: 9443 name: provisioner diff --git a/deploy/ibm/kustomization.yaml b/deploy/ibm/kustomization.yaml index 6efa42c3..3e524d1e 100644 --- a/deploy/ibm/kustomization.yaml +++ b/deploy/ibm/kustomization.yaml @@ -23,11 +23,31 @@ kind: Kustomization resources: - ../openshift +# --- remap external images to the cluster-internal registry --- +# ROKS worker nodes cannot reach quay.io, registry.access.redhat.com, ghcr.io, +# or docker.io; all images must be pulled from the internal registry. Mirror the +# platform images into the internal registry (see skills/deploy/ibm-cluster +# SKILL.md §5.2), then enable this block and set newName/newTag to match your +# mirrored image pullspecs in image-registry.openshift-image-registry.svc:5000. +# +# images: +# - name: quay.io/redhat-services-prod/hcm-eng-prod-tenant/hypershell-main/hypershell-api-server-main +# newName: image-registry.openshift-image-registry.svc:5000/hypershell/hypershell-api-server +# newTag: "" +# - name: quay.io/redhat-services-prod/hcm-eng-prod-tenant/hypershell-main/hypershell-control-plane-main +# newName: image-registry.openshift-image-registry.svc:5000/hypershell/hypershell-controller +# newTag: "" +# - name: registry.access.redhat.com/hi/postgresql +# newName: image-registry.openshift-image-registry.svc:5000/hypershell/postgresql +# newTag: "18" + patches: # --- select Route ingress mode for the control-plane reconciler --- - # Strategic-merge on the env list (merged by name): adds GATEWAY_INGRESS_MODE - # and overrides GATEWAY_API_BASE_DOMAIN. GATEWAY_API_GATEWAY_NAME is unused in - # route mode (the reconciler never looks up a shared Gateway) and is left as-is. + # Strategic-merge on the env list (merged by name): adds GATEWAY_INGRESS_MODE, + # overrides GATEWAY_API_BASE_DOMAIN, and sets per-tenant gateway database + + # sandbox-base images to point at the cluster-internal registry (which ROKS + # nodes can reach, unlike quay.io/ghcr.io/docker.io). GATEWAY_API_GATEWAY_NAME + # is unused in route mode and is left as-is. - patch: | apiVersion: apps/v1 kind: Deployment @@ -46,3 +66,23 @@ patches: # OVERRIDE: set to this cluster's ingress subdomain, e.g. # hysh-ibm-01--0000.us-east.containers.appdomain.cloud value: openshell.ibm.example.com + - name: HYPERSHELL_DATABASE_IMAGE + # OVERRIDE: set to the internal-registry pullspec for + # postgres:18 (mirror of docker.io/library/postgres:18). + # Nodes cannot pull docker.io; this image is served from the + # cluster-internal registry. See skills/deploy/ibm-cluster + # §5.2 for the mirror command. + value: image-registry.openshift-image-registry.svc:5000/openshift/postgres:18 + - name: GATEWAY_SANDBOX_IMAGE + # OVERRIDE: set to the internal-registry pullspec for the + # openshell-sandbox base image (mirror of + # ghcr.io/nvidia/openshell-community/sandboxes/base:latest). + # Nodes cannot pull ghcr.io; this is a prerequisite for + # sandbox pod creation. Without it, Sandbox CRs are admitted + # but pods ImagePullBackOff. See skills/deploy/ibm-cluster + # §5.2 for the mirror command. + value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-sandbox-base:latest + - name: GATEWAY_IMAGE + value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-gateway:v0.0.109-rhaiv.0 + - name: GATEWAY_SUPERVISOR_IMAGE + value: image-registry.openshift-image-registry.svc:5000/openshift/openshell-supervisor:v0.0.109-rhaiv.0 diff --git a/scripts/kind/lib.sh b/scripts/kind/lib.sh index fe39d840..0e945eba 100755 --- a/scripts/kind/lib.sh +++ b/scripts/kind/lib.sh @@ -35,7 +35,7 @@ fi if [[ "$(basename "${CONTAINER_ENGINE}")" == "podman" ]]; then export KIND_EXPERIMENTAL_PROVIDER=podman fi -: "${GATEWAY_IMAGE:=ghcr.io/nvidia/openshell/gateway:0.0.109}" +: "${GATEWAY_IMAGE:=quay.io/opendatahub/odh-openshell-gateway:v0.0.109-rhaiv.0@sha256:a80b79e514826e8d57ea137749cf18a6e7f3d92e26bfefe005f3a9c4a55b8bdd}" : "${KEYCLOAK_HOSTNAME:=keycloak.hypershell.localhost}" : "${KEYCLOAK_OIDC_ISSUER:=https://${KEYCLOAK_HOSTNAME}/realms/hypershell}" : "${KEYCLOAK_OIDC_CLIENT_ID:=hypershell-frontend}" diff --git a/skills/RECONCILE.md b/skills/RECONCILE.md index 0b168a41..7228737a 100644 --- a/skills/RECONCILE.md +++ b/skills/RECONCILE.md @@ -24,8 +24,10 @@ skills/ │ ├── full-stack-pipeline/ # Single-spec wave-based implementation pipeline │ └── dev-cluster/ # Kind cluster lifecycle for local testing ├── deploy/ -│ ├── deploy-cluster/ # OpenShift deployment (internal registry, kustomize) -│ └── kind/ # Kind local development (image loading, NodePort) +│ ├── cloud-hub-ingress-bootstrap/ # Shared Gateway API ingress per cloud hub +│ ├── deploy-cluster/ # OpenShift deployment (Keycloak, OIDC, CNPG, kustomize) +│ ├── gcp-cluster/ # GCP OSD cluster deployment (Route mode) +│ └── ibm-cluster/ # IBM ROKS cluster provisioning and deployment (Route mode) ├── plan/ │ └── spec/ # Spec authoring (desired state) ├── review/ @@ -36,10 +38,11 @@ skills/ ├── align/ # Convention compliance scoring ├── jira-log/ # Jira work logging ├── maintain-ci/ # CI and component registration maintenance - └── memory/ # Project memory management + ├── memory/ # Project memory management + └── update-openshell/ # Update to upstream OpenShell releases ``` -**SDLC flow**: `/reconcile` → `/spec` → `/full-stack-pipeline` → `/deploy-cluster` or `/kind` +**SDLC flow**: `/reconcile` → `/spec` → `/full-stack-pipeline` → `/deploy-cluster` or `/dev-cluster` --- @@ -716,3 +719,4 @@ label-selected pod informer. | 2026-08-21 | 361305e | HYPERSHELL-49 scoped gap analysis | 69% | Added 15 OpenShellGatewayServiceAccount requirements, all initially missing. Planned strict API -> SDK -> service/Keycloak -> CLI -> UI -> integration waves. Recorded the post-delivery token-verification contradiction without changing specs. | | 2026-08-21 | working tree | Executed HYPERSHELL-49 SA-W1..W3 | pending final recount | Added nested REST/OpenAPI, generated SDKs, durable persistence/audit, exact gateway-scoped Keycloak clients, one-time verified secret delivery, role-capped authorization, expiration/revoke/delete reconciliation, and gateway cleanup barriers. | | 2026-08-21 | working tree | Executed HYPERSHELL-49 SA-W4 | pending final recount | Extended the CLI generator for the nested gateway collection; added create/list/get/revoke/delete commands, explicit mode-0600 one-time credential output, expiration handling, workspace guidance, and secret-redaction tests. | +| 2026-09-01 | feecbcb, da771fb | Reconciled commit-driven stale doc gaps | 82% (unchanged) | Two recent commits removed hardcoded image defaults (`GATEWAY_IMAGE`/`GATEWAY_SUPERVISOR_IMAGE` now required env vars, no fallback) and unified deploy paths (deleted `components/api-server/deploy/*`, using repo-root `deploy/` as single source of truth). Updated 7 docs: `skills/deploy/ibm-cluster/SKILL.md` (image refs, path, namespace, image-var explanation), `skills/deploy/gcp-cluster/SKILL.md` (path fix, RBAC ref), `skills/deploy/deploy-cluster/SKILL.md` (full rewrite: Keycloak bootstrap, `hypershell-api-config` Secret creation, CNPG database, OIDC/JWT security, troubleshooting for missing Secret), `skills/tooling/update-openshell/SKILL.md` (grep patterns for new image names, search path fixes), `skills/RECONCILE.md` (skill directory tree, this log entry), `README.md` (env var rows, namespace refs), `specs/platform/openshift-development.spec.md` (deploy/ directory layout, overlay limitations note). Overlay gaps surfaced: `deploy/openshift/` requires manually-created `hypershell-api-config` Secret (missing from repo; documented in deploy-cluster), hardcoded domain placeholder, missing Keycloak Route on OpenShift. Marked as known limitations in specs. | diff --git a/skills/build/dev-cluster/SKILL.md b/skills/build/dev-cluster/SKILL.md index 87eb8f45..71ce524a 100644 --- a/skills/build/dev-cluster/SKILL.md +++ b/skills/build/dev-cluster/SKILL.md @@ -19,9 +19,10 @@ description: > ## Cluster Lifecycle ```bash -make kind-up # Create cluster with images -make kind-down # Destroy cluster -make kind-rebuild # Rebuild all + reload + restart +LOCAL_IMAGES=true make kind-up # Create cluster with local images +make kind-teardown # Destroy cluster +make kind-api-server-up # Build + swap API server from working tree +make kind-control-plane-up # Build + swap control plane from working tree make kind-status # Show cluster status ``` @@ -39,7 +40,7 @@ Map changed files to components: ### Step 2: Build and Deploy **If cluster doesn't exist:** `make kind-up` -**If cluster exists:** `make kind-rebuild` +**If cluster exists:** `make kind-api-server-up` (for API server) and/or `make kind-control-plane-up` (for controller) ### Step 3: Verify Deployment ```bash @@ -69,7 +70,8 @@ To teardown: make kind-down ### Pods in ImagePullBackOff Kind has no registry. Ensure `imagePullPolicy: IfNotPresent`: ```bash -make kind-rebuild +make kind-api-server-up +make kind-control-plane-up ``` ### Pods in CrashLoopBackOff @@ -80,16 +82,18 @@ kubectl describe pod -l app=