Problem: Intermittent SIGSEGV from DBL_MAX sentinel + NULL best_key (float ABI mismatch / NaN) —— Trove problem library

What you will run into

Real case: MariaDB 5.5.64 intermittently crashed on a domestic ARM64 server (early XinChuang era, CPU without FPU), SIGSEGV s=11;

at the crash site gdb showed best_key = 0x0 (null pointer) + best = DBL_MAX (1.797e+308 sentinel initial value, never overwritten by any candidate).

It looks "intermittent" because it only triggers when data/environment makes every comparison constantly false; normally everything works.

Why this happens

Any search/selection code using "DBL_MAX as the initial best + a pointer recording the best item" crashes once no comparison ever holds: the sentinel is never overwritten → the pointer stays NULL → dereferencing it crashes. Three layers of cause:

The crash chain is "ABI mismatch → NaN → comparison always false → sentinel never overwritten → null dereference":

at the environment layer, ARM64 systems without an FPU compile system components with the soft-float ABI while the MariaDB binary is hard-float — the float register conventions disagree, so values are read as NaN/garbage;

at the mechanism layer, IEEE 754 says NaN compares false against everything, so if (score < best) never holds for NaN, best stays at the initial DBL_MAX and best_key stays NULL;

at the code layer, there is no null check before use — best_key is exactly the "was there a candidate hit" validity sentinel.

All three layers are required: no ABI mismatch, no NaN; no NaN semantics, the sentinel gets overwritten; no code defect, no crash.

Deterministic takeaway: for search/selection code with float arithmetic crashing, suspect the NaN path first; defensive writing = filter with isfinite before comparing + null-check before dereferencing (if (best_key)); the tell is abnormal values (NaN/huge) rather than logic errors;

fixed diagnostic: NaN/Inf in logs or "crashes on first run" + domestic ARM environment = check the ABI layer first.

Step by step

1. Code safety net (should be done everywhere) — Required

Filter non-finite values before float comparisons:

if (!isnan(s) && !isinf(s) && s < best) { best = s; best_key = &cands[i]; }

Null-check the sentinel pointer before use:

if (best_key == NULL) { /* take the error path / return "no result" */ }

Add logging for best — if DBL_MAX ever appears, no comparison ever succeeded, pointing straight at the float layer.

2. Environment root fix: confirm the float ABI matches — Required

On XinChuang soft-float environments, rebuild the target program with -msoft-float (the article's fix). Diagnosis:

readelf -A <binary> | grep -i vfp
ldd <binary>
grep -i fpu /proc/cpuinfo

Check the ARM Tag_ABI_VFP_args flag, whether dependencies mixed in hard-float libraries, and whether the CPU really lacks an FPU.

3. Watch out for soft-float library behavior — Required

Soft-float/FTZ implementations often flush denormals to 0 (measured DBL_MIN/2 == 0), so tiny-value comparisons/branches differ from hard float; when the same code behaves differently under different ABIs, suspect the float ABI first, not the business logic.

How to confirm it works

The fixed version takes the error path on the same input without crashing (measured output shape):

isnan filter + best_key == NULL guard → same input takes the error path, no crash (measured #2)

Bug version: NaN score array → best stays DBL_MAX, best_key stays NULL → dereference SIGSEGV (measured #1, matching the article's dump item by item).

Other environments

Symptom

Real case: MariaDB 5.5.64 intermittently crashes on domestic ARM64 servers (early Xinchuang hardware without an FPU), SIGSEGV s=11; gdb shows best_key = 0x0 (NULL pointer) + best = DBL_MAX (the 1.797e+308 sentinel initial value, never overwritten by any candidate).

Generalized: any search/selection code using "DBL_MAX as initial best + a pointer recording the best item" crashes when no comparison ever holds (NaN involved, FPU instruction faults, soft/hard float ABI mismatch) — the sentinel never gets overwritten, the pointer stays NULL, and dereferencing crashes. It looks "intermittent" because it only happens when data/environment makes every comparison false.

Cause

1. Environment root cause (from the article): early Xinchuang ARM64 CPUs lack an FPU / float instructions fault → the system is compiled and run with a soft-float ABI (-msoft-float); the MariaDB binary is compiled with a hard-float ABI. A soft-float program loading hard-float binaries/libraries → float register conventions and results are misaligned → float values are read as NaN/garbage.
2. Mechanism root cause (general): under IEEE 754, NaN comparisons are always false (NaN < x, NaN >= x both false). if (score < best) never holds → best stays DBL_MAX and best_key stays NULL.
3. Code defect: no null check before use — best_key itself is the "did any candidate match" validity sentinel, but the code dereferences it directly without an if (!best_key) guard.

Solution

1. Root fix (environment): confirm the float ABI matches. On soft-float environments recompile the target with -msoft-float (the article's fix); diagnosis: readelf -A <binary> for the ARM Tag_ABI_VFP_args attribute, ldd to check for mixed hard-float dependencies, dmesg | grep -i fpu / grep -i fpu /proc/cpuinfo to confirm the CPU really lacks an FPU.
2. Belt-and-suspenders (code, do it on every platform):

• Filter non-finite values before comparisons: if (!isnan(s) && !isinf(s) && s < best) — NaN/Inf never participates in "best" selection;
• Null-check the sentinel pointer before use: if (best_key == NULL) { error path / "no result" }, never dereference directly;
• Optional assertion aid: log best — if DBL_MAX shows up, comparisons never succeeded, pointing straight at the float layer.

1. Beware soft-float library behavior: soft-float/FTZ implementations often flush denormals to zero (see evidence 3), making tiny-value comparisons/branches differ from hard float; when the same code behaves differently across ABIs, suspect the float ABI before the business logic.