Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 5b201d7e authored by Anthony Stange's avatar Anthony Stange
Browse files

Requantize by resolution / 8 

Requantizing by resolution / 4 causing the calibration error / min on
calibrated sensors to be too high. Changing to 8 reduces that error
enough to keep sensor accuracy below our thresholds.

Bug: 158451802
Test: Verify that calibration error / min is low
Change-Id: I350980734cc0a46f4f06878371e0d9bd7ce07de0
parent 73b522de

services/sensorservice/SensorDeviceUtils.h

+1 −1
Original line number Original line Diff line number Diff line
@@ -36,7 +36,7 @@ inline void quantizeValue(float *value, double resolution) { 
    // Increase the value of the sensor's nominal resolution to ensure that

    // Increase the value of the sensor's nominal resolution to ensure that

    // sensor accuracy improvements, like runtime calibration, are not masked

    // sensor accuracy improvements, like runtime calibration, are not masked

    // during requantization.

    // during requantization.

    double incRes = 0.25 * resolution;

    double incRes = 0.125 * resolution;

    *value = round(static_cast<double>(*value) / incRes) * incRes;

    *value = round(static_cast<double>(*value) / incRes) * incRes;

}

}