mirrors/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2025-11-04 02:30:34 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

net: introduce DST_NOCACHE flag

Browse source 
While doing stress tests with IP route cache disabled, and multi queue
devices, I noticed a very high contention on one rwlock used in
neighbour code.

When many cpus are trying to send frames (possibly using a high
performance multiqueue device) to the same neighbour, they fight for the
neigh->lock rwlock in order to call neigh_hh_init(), and fight on
hh->hh_refcnt (a pair of atomic_inc/atomic_dec_and_test())

But we dont need to call neigh_hh_init() for dst that are used only
once. It costs four atomic operations at least, on two contended cache
lines, plus the high contention on neigh->lock rwlock.

Introduce a new dst flag, DST_NOCACHE, that is set when dst was not
inserted in route cache.

With the stress test bench, sending 160000000 frames on one neighbour,
results are :

Before patch:

real	2m28.406s
user	0m11.781s
sys	36m17.964s


After patch:

real	1m26.532s
user	0m12.185s
sys	20m3.903s

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This commit is contained in:
Eric Dumazet 2010-10-03 22:17:54 -07:00 committed by David S. Miller
parent 9a7241c21b
commit c7d4426a98
3 changed files with 9 additions and 5 deletions
repo.diff.show_diff_stats Download patch file Download diff file Expand all files Collapse all files

9
include/net/dst.h
View file

@ -43,10 +43,11 @@ struct dst_entry {
short error; short error;
short obsolete; short obsolete;
int flags; int flags;
#define DST_HOST 1 #define DST_HOST 0x0001
#define DST_NOXFRM 2 #define DST_NOXFRM 0x0002
#define DST_NOPOLICY 4 #define DST_NOPOLICY 0x0004
#define DST_NOHASH 8 #define DST_NOHASH 0x0008
#define DST_NOCACHE 0x0010
unsigned long expires; unsigned long expires;
unsigned short header_len; /* more space at head required */ unsigned short header_len; /* more space at head required */

4
net/core/neighbour.c
View file

@ -1210,7 +1210,9 @@ int neigh_resolve_output(struct sk_buff *skb)
if (!neigh_event_send(neigh, skb)) { if (!neigh_event_send(neigh, skb)) {
int err; int err;
struct net_device *dev = neigh->dev; struct net_device *dev = neigh->dev;
if (dev->header_ops->cache && !dst->hh) { if (dev->header_ops->cache &&
!dst->hh &&
!(dst->flags & DST_NOCACHE)) {
write_lock_bh(&neigh->lock); write_lock_bh(&neigh->lock);
if (!dst->hh) if (!dst->hh)
neigh_hh_init(neigh, dst, dst->ops->protocol); neigh_hh_init(neigh, dst, dst->ops->protocol);

1
net/ipv4/route.c
View file

@ -1107,6 +1107,7 @@ static int rt_intern_hash(unsigned hash, struct rtable *rt,
* on the route gc list. * on the route gc list.
*/ */
rt->dst.flags |= DST_NOCACHE;
if (rt->rt_type == RTN_UNICAST || rt->fl.iif == 0) { if (rt->rt_type == RTN_UNICAST || rt->fl.iif == 0) {
int err = arp_bind_neighbour(&rt->dst); int err = arp_bind_neighbour(&rt->dst);
if (err) { if (err) {