vms-1fc: widen the libvmssys raw-syscall path to 64-bit (LLP64 /dev/vms ioctl pointer fix) - #1064
Merged
Merged
Conversation
…pointer fix) On the alpha-dec-vms C model `long`/`unsigned long` are 32 bits while pointers are 64 (LLP64), so the libvmssys raw-syscall path TRUNCATED every pointer passed to the /dev/vms transport: the request-block pointer to ioctl(/dev/vms, ...) lost its high half and the executive's copy_from_user read a garbage address. This is a real bug on the alpha VMS-native backend (LIBVMSSYS$SHR / the kif transport / LIBVMSRMS$SHR), latent until a runtime path actually drove an RMS-over-ACP call. Widen the whole raw-syscall path to a guaranteed-64-bit `vms_reg_t` (== long long): - vms_syscall.h: __vms_syscall0..6 params + return; every vms_sys_* pointer cast; and vms_sys_ioctl's `arg` param (unsigned long -> vms_reg_t). - arch/alpha/syscall_vms.c: the EVAX callsys trampolines match (long long). - kif_transport_linux.c: cast the request-block pointer through vms_reg_t, not `unsigned long` -- THIS is the /dev/vms pointer that was truncating. - vms_bgsock.c: widen its ioctl pointer casts to match. 3-way safe / non-regressing: - x86_64 / aarch64 / alpha-linux-gnu are LP64: `long long` == `long` in width and ABI, so the codegen is byte-identical -- a strict no-op there. - The actual fix is only on the alpha-dec-vms cc1 (LLP64). - VAX is untouched by construction: it takes the __NetBSD__ branch (arch/vax/vms_syscall_netbsd.h) and compiles NONE of these declarations. Proven here: compiles clean on x86_64 (-Wall -Wextra), vms_reg_t is 64-bit, and the alpha-dec-vms cross build links zero-deferred with it (LIBVMSRMS$SHR + the producer graph + the veneer image). Its runtime purpose -- 64-bit /dev/vms ioctl pointers actually flowing to the ACP -- gets its final exercise when the CRTL->RMS veneer image runs on the executive (rung 4, vms-f49, tracked separately). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJZf62TMXxvy6fXzFQYfLQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On the alpha-dec-vms C model
long/unsigned longare 32 bits while pointers are 64 (LLP64), so the libvmssys raw-syscall path truncated every pointer passed to the/dev/vmstransport — the request-block pointer toioctl(/dev/vms, …, &kif_request)lost its high half and the executive'scopy_from_userread a garbage address. A real bug on the alpha VMS-native backend (LIBVMSSYS$SHR→ the kif transport →LIBVMSRMS$SHR), latent until a runtime path actually drove an RMS-over-ACP call.The fix — one guaranteed-64-bit register word
vms_syscall.h:__vms_syscall0..6params + return, everyvms_sys_*pointer cast, andvms_sys_ioctl'sargparam (unsigned long→vms_reg_t), all widened tovms_reg_t(==long long).arch/alpha/syscall_vms.c: the EVAXcallsystrampolines match (long long).kif_transport_linux.c: cast the request-block pointer throughvms_reg_t, notunsigned long— THIS is the/dev/vmspointer that was truncating.vms_bgsock.c: widen its ioctl pointer casts to match.3-way safe / non-regressing
long long==longin width and ABI, so the codegen is byte-identical — a strict no-op there.__NetBSD__branch (arch/vax/vms_syscall_netbsd.h) and compiles none of these declarations.Proven
Compiles clean on x86_64 (
-Wall -Wextra);vms_reg_tis 64-bit; the alpha-dec-vms cross build links zero-deferred with it (LIBVMSRMS$SHR+ the producer graph + the veneer image). Honest scope: this is build/link/activation-proven and non-regressing across alpha / VAX / x86_64. Its runtime purpose — 64-bit/dev/vmsioctl pointers actually flowing to the ACP — gets its final exercise when the CRTL→RMS veneer image runs on the executive (rung 4, vms-f49, tracked separately in #1063).🤖 Generated with Claude Code