Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Commit 9b3907c5 authored by Eric Dumazet's avatar Eric Dumazet Committed by Greg Kroah-Hartman
Browse files

rfs: annotate lockless accesses to sk->sk_rxhash 



[ Upstream commit 1e5c647c3f6d4f8497dedcd226204e1880e0ffb3 ]

Add READ_ONCE()/WRITE_ONCE() on accesses to sk->sk_rxhash.

This also prevents a (smart ?) compiler to remove the condition in:

if (sk->sk_rxhash != newval)
	sk->sk_rxhash = newval;

We need the condition to avoid dirtying a shared cache line.

Fixes: fec5e652 ("rfs: Receive Flow Steering")
Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
Reviewed-by: default avatarSimon Horman <simon.horman@corigine.com>
Reviewed-by: default avatarKuniyuki Iwashima <kuniyu@amazon.com>
Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
Signed-off-by: default avatarSasha Levin <sashal@kernel.org>
parent 8a200067

include/net/sock.h

+13 −5
Original line number Diff line number Diff line
@@ -988,8 +988,12 @@ static inline void sock_rps_record_flow(const struct sock *sk) 
		 * OR	an additional socket flag

		 * [1] : sk_state and sk_prot are in the same cache line.

		 */

		if (sk->sk_state == TCP_ESTABLISHED)

			sock_rps_record_flow_hash(sk->sk_rxhash);

		if (sk->sk_state == TCP_ESTABLISHED) {

			/* This READ_ONCE() is paired with the WRITE_ONCE()

			 * from sock_rps_save_rxhash() and sock_rps_reset_rxhash().

			 */

			sock_rps_record_flow_hash(READ_ONCE(sk->sk_rxhash));

		}

	}

#endif

}
@@ -998,15 +1002,19 @@ static inline void sock_rps_save_rxhash(struct sock *sk, 
					const struct sk_buff *skb)

{

#ifdef CONFIG_RPS

	if (unlikely(sk->sk_rxhash != skb->hash))

		sk->sk_rxhash = skb->hash;

	/* The following WRITE_ONCE() is paired with the READ_ONCE()

	 * here, and another one in sock_rps_record_flow().

	 */

	if (unlikely(READ_ONCE(sk->sk_rxhash) != skb->hash))

		WRITE_ONCE(sk->sk_rxhash, skb->hash);

#endif

}



static inline void sock_rps_reset_rxhash(struct sock *sk)

{

#ifdef CONFIG_RPS

	sk->sk_rxhash = 0;

	/* Paired with READ_ONCE() in sock_rps_record_flow() */

	WRITE_ONCE(sk->sk_rxhash, 0);

#endif

}