Skip to content

fixedpoint: avoid signed-overflow UB in scalar Neg - #219

Open
EylonKrause wants to merge 1 commit into
google:masterfrom
EylonKrause:fix/scalar-neg-intmin-ub
Open

fixedpoint: avoid signed-overflow UB in scalar Neg#219
EylonKrause wants to merge 1 commit into
google:masterfrom
EylonKrause:fix/scalar-neg-intmin-ub

Conversation

@EylonKrause

Copy link
Copy Markdown

Problem

The scalar Neg primitive in fixedpoint/fixedpoint.h computes -a, which is
signed-integer overflow (undefined behavior) when a is the most negative
representable value (-2147483648 for int32, -32768 for int16).

It is reachable from the public tanh() and logistic() functions. Both
compute SelectUsingMask(mask_if_..., a, -a); because C++ evaluates function
arguments unconditionally, -a (hence Neg(INT_MIN)) is always executed — even
for the minimum-valued FixedPoint input, whose negated value is then masked
away.

Building the existing test/test_fixedpoint.cc with UBSan reproduces it:

$ g++ -std=c++11 -O2 -I. -fsanitize=address,undefined test/test_fixedpoint.cc -o t -lpthread && ./t
fixedpoint/fixedpoint.h:113:11: runtime error: negation of -2147483648 cannot be represented in type 'int'
    #0 gemmlowp::Neg<int>(int)               fixedpoint.h:113
    #1 gemmlowp::operator-<int, 0>(...)       fixedpoint.h:594
    #2 gemmlowp::tanh<int, 0>(...)            fixedpoint.h:844
    #3 Op<int>                                test/test_fixedpoint.cc:280

Fix

Negate through the unsigned counterpart type, where the operation is well
defined, then convert back:

typedef typename std::make_unsigned<tIntegerType>::type UnsignedType;
return static_cast<tIntegerType>(-static_cast<UnsignedType>(a));

This:

  • removes the UB while producing a bit-identical result (the value wraps
    back to the same most-negative value, exactly as the previous code did in
    practice);
  • matches every SIMD implementation of Neg (vnegq_s32, _mm_sign_epi32,
    _mm256_sign_epi32, MSA subv, wasm_i32x4_neg), which already negate
    without UB;
  • follows the UB-avoidance policy already documented on ShiftLeft in the same
    file ("no Undefined Behavior, but the results are implementation-defined").

The unsigned→signed conversion back is implementation-defined (not UB) in
C++11/14/17 and well-defined in C++20; the same in-practice-consistent behavior
is already relied on by ShiftLeft and ShiftRight in this file.

Testing

test/test_fixedpoint.cc now passes cleanly under
-fsanitize=address,undefined (previously it aborted on the UBSan error above):
PASS (Scalar int32) / PASS (Scalar int16). Outputs for the minimum input
are unchanged and match their float references, e.g. tanh(-1.0) ≈ -0.7616,
logistic(-1.0) ≈ 0.2689.

Disclosure: this contribution was authored with an AI coding assistant (Claude) and reviewed before submission.

The scalar `Neg` primitive computed `-a`, which is undefined behavior when `a`
is the most negative representable value (e.g. -2147483648 for int32).

This is reachable from the public `tanh()` and `logistic()` functions: both
compute `SelectUsingMask(mask, a, -a)`, and C++ evaluates the `-a` argument
unconditionally, so `Neg(INT_MIN)` runs for the minimum-valued FixedPoint input
even though its result is then masked out. Building the existing
test/test_fixedpoint.cc under UBSan reports:

  fixedpoint.h:113:11: runtime error: negation of -2147483648 cannot be
  represented in type 'int'

Negate through the unsigned counterpart type instead, where the operation is
well-defined and wraps back to the same value. This matches every SIMD
implementation of Neg (vnegq_s32, _mm_sign_epi32, _mm256_sign_epi32, MSA subv,
wasm_i32x4_neg), which negate in hardware without UB, and follows the
UB-avoidance policy already documented on ShiftLeft. No functional change: the
wrapped result is bit-identical to the previous in-practice behavior, and
test_fixedpoint now passes cleanly under -fsanitize=address,undefined.
@wowo68

wowo68 commented Aug 21, 2026 via email

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants