nagai/kernel
3
0
Fork 0
forked from mirrors/linux
Code Pull requests Activity

udp: avoid calling sock_def_readable() if possible

Browse source 
sock_def_readable() is quite expensive (particularly
when ep_poll_callback() is in the picture).

We must call sk->sk_data_ready() when :

- receive queue was empty, or
- SO_PEEK_OFF is enabled on the socket, or
- sk->sk_data_ready is not sock_def_readable.

We still need to call sk_wake_async().

Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Link: https://lore.kernel.org/r/20240328144032.1864988-4-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This commit is contained in:
Eric Dumazet 2024-03-28 14:40:31 +00:00 committed by Jakub Kicinski
parent 6a1f12dd85
commit 612b1c0dec
1 changed files with 11 additions and 3 deletions
repo.diff.show_diff_stats Download patch file Download diff file Expand all files Collapse all files

14
net/ipv4/udp.c
View file

@ -1492,6 +1492,7 @@ int __udp_enqueue_schedule_skb(struct sock *sk, struct sk_buff *skb)
struct sk_buff_head *list = &sk->sk_receive_queue;
int rmem, err = -ENOMEM;
spinlock_t *busy = NULL;
bool becomes_readable;
int size, rcvbuf;
/* Immediately drop when the receive queue is full.
@ -1532,12 +1533,19 @@ int __udp_enqueue_schedule_skb(struct sock *sk, struct sk_buff *skb)
*/
sock_skb_set_dropcount(sk, skb);
becomes_readable = skb_queue_empty(list);
__skb_queue_tail(list, skb);
spin_unlock(&list->lock);
if (!sock_flag(sk, SOCK_DEAD))
INDIRECT_CALL_1(sk->sk_data_ready, sock_def_readable, sk);
if (!sock_flag(sk, SOCK_DEAD)) {
if (becomes_readable ||
sk->sk_data_ready != sock_def_readable ||
READ_ONCE(sk->sk_peek_off) >= 0)
INDIRECT_CALL_1(sk->sk_data_ready,
sock_def_readable, sk);
else
sk_wake_async(sk, SOCK_WAKE_WAITD, POLL_IN);
}
busylock_release(busy);
return 0;