Skip to content

Retry phone-home while the backend binding is pending; fail the job if the handshake never lands #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rcooney-sh merged 1 commit into from
Sep 16, 2026
Merged

Retry phone-home while the backend binding is pending; fail the job if the handshake never lands #13

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 59 additions & 30 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
name: 'Begin Testbox'
description: 'Initialize a Blacksmith Testbox session. Phones home to receive config, installs SSH key and rsync.'

Check warning on line 2 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

2:81 [line-length] line too long (114 > 80 characters)

inputs:
testbox_id:
description: 'Testbox session ID (from workflow_dispatch input). Leave empty for validation mode.'

Check warning on line 6 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

6:81 [line-length] line too long (102 > 80 characters)
required: false
default: ''
api_url:
description: 'Override the backend API URL (default: discovered from VM metadata)'

Check warning on line 10 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

10:81 [line-length] line too long (86 > 80 characters)
required: false

runs:
Expand All @@ -17,7 +17,7 @@
shell: bash
run: |
if [ -z "${{ inputs.testbox_id }}" ]; then
echo "::notice::Running in validation mode — setup steps will be validated without starting a testbox session."

Check warning on line 20 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

20:81 [line-length] line too long (121 > 80 characters)
fi

- name: Discover metadata service
Expand All @@ -25,22 +25,22 @@
shell: bash
run: |
if [ -z "${BLACKSMITH_AGENT_ADDR:-}" ]; then
echo "::warning::BLACKSMITH_AGENT_ADDR is not set; testbox is only supported on Blacksmith runners. Skipping testbox setup."

Check warning on line 28 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

28:81 [line-length] line too long (134 > 80 characters)
echo "available=false" >> "$GITHUB_OUTPUT"
exit 0
fi
METADATA_PORT="${METADATA_PORT:-}"
if [ -z "$METADATA_PORT" ]; then
METADATA_PORT=$(tr ' ' '\n' < /proc/cmdline | grep '^metadata_port=' | cut -d'=' -f2 || true)

Check warning on line 34 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

34:81 [line-length] line too long (103 > 80 characters)
if [ -z "$METADATA_PORT" ]; then
echo "::warning::metadata_port not found in kernel cmdline; testbox is only supported on Blacksmith runners. Skipping testbox setup."

Check warning on line 36 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

36:81 [line-length] line too long (145 > 80 characters)
echo "available=false" >> "$GITHUB_OUTPUT"
exit 0
fi
export METADATA_PORT
fi
echo "available=true" >> "$GITHUB_OUTPUT"
echo "metadata_addr=${BLACKSMITH_AGENT_ADDR}:${METADATA_PORT}" >> "$GITHUB_OUTPUT"

Check warning on line 43 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

43:81 [line-length] line too long (90 > 80 characters)

- name: Phone home and configure testbox
if: steps.metadata.outputs.available == 'true' && inputs.testbox_id
Expand All @@ -54,22 +54,23 @@
mkdir -p "$STATE"
chmod 700 "$STATE"

INSTALLATION_MODEL_ID=$(curl -s --connect-timeout 2 --max-time 5 "http://${METADATA_ADDR}/installationModelID")

Check warning on line 57 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

57:81 [line-length] line too long (119 > 80 characters)
if [ -n "$API_URL_OVERRIDE" ]; then
API_URL="$API_URL_OVERRIDE"
else
API_URL=$(curl -s --connect-timeout 2 --max-time 5 "http://${METADATA_ADDR}/backendURL")

Check warning on line 61 in action.yml

View workflow job for this annotation

GitHub Actions / validate 

61:81 [line-length] line too long (98 > 80 characters)
fi

# Piggyback on the VM's stickyDiskToken for phone-home authentication.
# This is a Sanctum token that proves the caller is a real Blacksmith VM
# belonging to the claimed installation — it's only accessible from inside
# the VM via the metadata service and is not publicly visible.
# It proves the caller is a real Blacksmith VM belonging to the claimed
# installation, and the backend binds it to the runner that adopted this
# testbox's run. It's only accessible from inside the VM via the metadata
# service and is not publicly visible.
AUTH_TOKEN=$(curl -s --connect-timeout 2 --max-time 5 "http://${METADATA_ADDR}/stickyDiskToken")

if [ -z "$API_URL" ] || [ -z "$INSTALLATION_MODEL_ID" ] || [ -z "$AUTH_TOKEN" ]; then
echo "Warning: could not read required metadata (backendURL, installationModelID, stickyDiskToken)"
exit 0
echo "::error::Could not read required metadata (backendURL, installationModelID, stickyDiskToken); the testbox cannot phone home."
exit 1
fi

if [ -n "$BLACKSMITH_HOSTNAME" ]; then
Expand All @@ -79,22 +80,55 @@
fi
RUNNER_SSH_PORT="${BLACKSMITH_SSH_PORT:-22}"

RESPONSE=$(curl -s -f -X POST "${API_URL}/api/testbox/phone-home" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${AUTH_TOKEN}" \
-d "{
\"testbox_id\": \"${TESTBOX_ID}\",
\"installation_model_id\": ${INSTALLATION_MODEL_ID},
\"status\": \"hydrating\",
\"ip_address\": \"${RUNNER_HOST}\",
\"ssh_port\": \"${RUNNER_SSH_PORT}\",
\"working_directory\": \"${GITHUB_WORKSPACE}\",
\"adopted_run_id\": \"${GITHUB_RUN_ID}\",
\"metadata\": {}
}" 2>/dev/null) || {
echo "Warning: phone-home (hydrating) failed, continuing in degraded mode"
RESPONSE=""
}
# The first phone-home can race warmup's write of the dispatched run ID
# and the adoption webhook. The backend answers 503 + Retry-After while
# that binding is pending, so retry transient failures up to a deadline
# and fail the job if the handshake never lands: without it the CLI's
# SSH key is never installed and the testbox is unusable.
PHONE_HOME_DEADLINE=$(( $(date +%s) + 90 ))
PHONE_HOME_RETRY_DELAY=2
PHONE_HOME_RESPONSE_FILE="$STATE/phone_home_response"
ATTEMPT=0
while true; do
ATTEMPT=$((ATTEMPT + 1))
HTTP_CODE=$(curl -s -o "$PHONE_HOME_RESPONSE_FILE" -w '%{http_code}' --max-time 30 \
-X POST "${API_URL}/api/testbox/phone-home" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${AUTH_TOKEN}" \
-d "{
\"testbox_id\": \"${TESTBOX_ID}\",
\"installation_model_id\": ${INSTALLATION_MODEL_ID},
\"status\": \"hydrating\",
\"ip_address\": \"${RUNNER_HOST}\",
\"ssh_port\": \"${RUNNER_SSH_PORT}\",
\"working_directory\": \"${GITHUB_WORKSPACE}\",
\"adopted_run_id\": \"${GITHUB_RUN_ID}\",
\"metadata\": {}
}") || HTTP_CODE=000

case "$HTTP_CODE" in
2??)
break
;;
000|408|425|429|5??)
;;
*)
echo "::error::Testbox phone-home rejected (HTTP ${HTTP_CODE}): $(cat "$PHONE_HOME_RESPONSE_FILE" 2>/dev/null)"
exit 1
;;
esac

if [ "$(date +%s)" -ge "$PHONE_HOME_DEADLINE" ]; then
echo "::error::Testbox phone-home did not succeed before the deadline (last HTTP ${HTTP_CODE}, ${ATTEMPT} attempts): $(cat "$PHONE_HOME_RESPONSE_FILE" 2>/dev/null)"
exit 1

@cursor cursor Bot Sep 16, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Later steps run after handshake failure

Low Severity

Handshake failures now exit 1 before writing ssh_public_key, but later steps still run because their if does not include success(). The SSH install step then fails on a missing file, adding a misleading error after the real phone-home failure.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e38ebc1. Configure here.

@blacksmith-staging blacksmith-staging Bot Sep 16, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

False positive: a step if without a status-check function implicitly includes success(), so the later steps are skipped once the phone-home step exits 1. The SSH install step also uses cat ... 2>/dev/null behind an [ -n ] guard, so a missing key file cannot produce a failure there.

@blacksmith-sh blacksmith-sh Bot Sep 16, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Composite action steps with an if that lacks a status function are implicitly wrapped in success() && (...), so these steps are skipped once phone-home exits 1. The SSH install step also tolerates a missing key file (cat ... 2>/dev/null guarded by [ -n ]), so it cannot emit a misleading error.

fi

echo "Phone-home not accepted yet (HTTP ${HTTP_CODE}, attempt ${ATTEMPT}); retrying in ${PHONE_HOME_RETRY_DELAY}s"
sleep "$PHONE_HOME_RETRY_DELAY"
done

RESPONSE=$(cat "$PHONE_HOME_RESPONSE_FILE")
rm -f "$PHONE_HOME_RESPONSE_FILE"

echo "$TESTBOX_ID" > "$STATE/testbox_id"
echo "$INSTALLATION_MODEL_ID" > "$STATE/installation_model_id"
Expand All @@ -105,16 +139,11 @@
echo "$GITHUB_WORKSPACE" > "$STATE/working_directory"
echo "$GITHUB_RUN_ID" > "$STATE/adopted_run_id"

if [ -n "$RESPONSE" ]; then
echo "$RESPONSE" | jq -r '.ssh_public_key // empty' > "$STATE/ssh_public_key"
IDLE_TIMEOUT=$(echo "$RESPONSE" | jq -r '.idle_timeout // empty')
echo "${IDLE_TIMEOUT:-10}" > "$STATE/idle_timeout"
else
echo "" > "$STATE/ssh_public_key"
echo "10" > "$STATE/idle_timeout"
fi
echo "$RESPONSE" | jq -r '.ssh_public_key // empty' > "$STATE/ssh_public_key"
IDLE_TIMEOUT=$(echo "$RESPONSE" | jq -r '.idle_timeout // empty')
echo "${IDLE_TIMEOUT:-10}" > "$STATE/idle_timeout"

echo "Testbox initialized: testbox_id=${TESTBOX_ID}"
echo "Testbox initialized: testbox_id=${TESTBOX_ID} (phone-home accepted after ${ATTEMPT} attempt(s))"

- name: Start heartbeat
if: steps.metadata.outputs.available == 'true' && inputs.testbox_id
Expand Down
Loading