Bluetooth: Improve RFCOMM __test_pf macro robustness

Value returned by this macro might be used as bit value so it should
return either 0 or 1 to avoid possible bugs (similar to NSC bug)
when shifting it.

Signed-off-by: Szymon Janc <szymon.janc@tieto.com>
Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>