nagai/kernel
3
0
Fork 0
forked from mirrors/linux
Code Pull requests Activity

blk-mq: bitmap tag: fix race on blk_mq_bitmap_tags::wake_cnt

Browse source 
This piece of code in bt_clear_tag() function is racy:

	bs = bt_wake_ptr(bt);
	if (bs && atomic_dec_and_test(&bs->wait_cnt)) {
		atomic_set(&bs->wait_cnt, bt->wake_cnt);
 		wake_up(&bs->wait);
	}

Since nothing prevents bt_wake_ptr() from returning the very
same 'bs' address on multiple CPUs, the following scenario is
possible:

    CPU1                                CPU2
    ----                                ----

0.  bs = bt_wake_ptr(bt);               bs = bt_wake_ptr(bt);
1.  atomic_dec_and_test(&bs->wait_cnt)
2.                                      atomic_dec_and_test(&bs->wait_cnt)
3.  atomic_set(&bs->wait_cnt, bt->wake_cnt);

If the decrement in [1] yields zero then for some amount of time
the decrement in [2] results in a negative/overflow value, which
is not expected. The follow-up assignment in [3] overwrites the
invalid value with the batch value (and likely prevents the issue
from being severe) which is still incorrect and should be a lesser.

Cc: Ming Lei <tom.leiming@gmail.com>
Cc: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
This commit is contained in:
Alexander Gordeev 2014-06-12 17:05:37 +02:00 committed by Jens Axboe
parent 8537b12034
commit 2971c35f35
1 changed files with 12 additions and 2 deletions
repo.diff.show_diff_stats Download patch file Download diff file Expand all files Collapse all files

14
block/blk-mq-tag.c
View file

@ -344,6 +344,7 @@ static void bt_clear_tag(struct blk_mq_bitmap_tags *bt, unsigned int tag)
{ {
const int index = TAG_TO_INDEX(bt, tag); const int index = TAG_TO_INDEX(bt, tag);
struct bt_wait_state *bs; struct bt_wait_state *bs;
int wait_cnt;
/* /*
* The unlock memory barrier need to order access to req in free * The unlock memory barrier need to order access to req in free
@ -352,10 +353,19 @@ static void bt_clear_tag(struct blk_mq_bitmap_tags *bt, unsigned int tag)
clear_bit_unlock(TAG_TO_BIT(bt, tag), &bt->map[index].word); clear_bit_unlock(TAG_TO_BIT(bt, tag), &bt->map[index].word);
bs = bt_wake_ptr(bt); bs = bt_wake_ptr(bt);
if (bs && atomic_dec_and_test(&bs->wait_cnt)) { if (!bs)
atomic_set(&bs->wait_cnt, bt->wake_cnt); return;
wait_cnt = atomic_dec_return(&bs->wait_cnt);
if (wait_cnt == 0) {
wake:
atomic_add(bt->wake_cnt, &bs->wait_cnt);
bt_index_atomic_inc(&bt->wake_index); bt_index_atomic_inc(&bt->wake_index);
wake_up(&bs->wait); wake_up(&bs->wait);
} else if (wait_cnt < 0) {
wait_cnt = atomic_inc_return(&bs->wait_cnt);
if (!wait_cnt)
goto wake;
} }
} }