Handle Clock Skew

[psteinroe]

Goal

Make cron evaluation use the database server's notion of time rather than assuming every worker host clock is synchronized with PostgreSQL.

PostgreSQL already decides whether executions are due through _private_current_time(). This issue aligns JavaScript cron calculation with that clock.

Design

Each worker maintains a database clock offset:

offset = database_time - midpoint(local_request_start, local_response_end)
adjusted_now = local_now + offset

Measure the offset:

• during worker startup, before calculating or registering cron occurrences;
• every 10 minutes while the worker is running.

Using the request midpoint compensates for symmetric query latency better than comparing the database timestamp with either the request start or response time.

Behavior

• Centralize adjusted clock access instead of applying offsets ad hoc in cron code.
• Pass the adjusted current time into cron-parser whenever the next occurrence is calculated.
• Keep PostgreSQL as the authority for due checks and execution timestamps.
• Store offset state per worker/orchestrator process; it is not persisted in task rows.
• Replace the offset atomically after a successful sample.
• If refresh fails, retain the last successful offset, log/report the failure, and keep the worker running.
• Stop the refresh timer during worker shutdown.
• Do not rewrite already-created executions when the measured offset changes.

The correction applies to initial cron registration and to the next occurrence scheduled while processing a cron execution.

Testability

Make the local clock and offset sampler injectable or otherwise controllable in tests. Tests must not rely on changing the machine clock.

Continue using pgconductor.fake_now as the database-side test clock where appropriate.

Acceptance tests

• Positive host clock skew produces the same next cron slot as database time.
• Negative host clock skew produces the same next cron slot as database time.
• Midpoint calculation accounts for request latency.
• Startup sampling completes before the first cron calculation.
• Offset refresh occurs every 10 minutes.
• A successful refresh affects subsequent calculations only.
• A failed refresh retains the previous offset and does not stop task processing.
• Multiple workers with different local skew derive the same cron dedupe key/slot.
• Existing no-skew and fake-time cron tests continue to pass.

Because the project is prerelease, update the current implementation and tests directly; no compatibility behavior is needed.