mirrors/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2025-11-03 18:20:25 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

radix-tree: free up the bottom bit of exceptional entries for reuse

Browse source 
We are guaranteed that pointers to radix_tree_nodes always have the
bottom two bits clear (because they come from a slab cache, and slab
caches have a minimum alignment of sizeof(void *)), so we can redefine
'radix_tree_is_internal_node' to only return true if the bottom two bits
have value '01'.  This frees up one quarter of the potential values for
use by the user.

Idea from Neil Brown.

Signed-off-by: Matthew Wilcox <willy@linux.intel.com>
Suggested-by: Neil Brown <neilb@suse.de>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Kirill Shutemov <kirill.shutemov@linux.intel.com>
Cc: Jan Kara <jack@suse.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This commit is contained in:
Matthew Wilcox 2016-05-20 17:03:54 -07:00 committed by Linus Torvalds
parent 78a9be0a0a
commit 3bcadd6fa6
1 changed files with 22 additions and 14 deletions
repo.diff.show_diff_stats Download patch file Download diff file Expand all files Collapse all files

36
include/linux/radix-tree.h
View file

@ -29,28 +29,37 @@
#include <linux/rcupdate.h> #include <linux/rcupdate.h>
/* /*
* Entries in the radix tree have the low bit set if they refer to a * The bottom two bits of the slot determine how the remaining bits in the
* radix_tree_node. If the low bit is clear then the entry is user data. * slot are interpreted:
* *
* We also use the low bit to indicate that the slot will be freed in the * 00 - data pointer
* next RCU idle period, and users need to re-walk the tree to find the * 01 - internal entry
* new slot for the index that they were looking for. See the comment in * 10 - exceptional entry
* radix_tree_shrink() for details. * 11 - locked exceptional entry
*
* The internal entry may be a pointer to the next level in the tree, a
* sibling entry, or an indicator that the entry in this slot has been moved
* to another location in the tree and the lookup should be restarted. While
* NULL fits the 'data pointer' pattern, it means that there is no entry in
* the tree for this index (no matter what level of the tree it is found at).
* This means that you cannot store NULL in the tree as a value for the index.
*/ */
#define RADIX_TREE_INTERNAL_NODE 1 #define RADIX_TREE_ENTRY_MASK 3UL
#define RADIX_TREE_INTERNAL_NODE 1UL
/* /*
* A common use of the radix tree is to store pointers to struct pages; * Most users of the radix tree store pointers but shmem/tmpfs stores swap
* but shmem/tmpfs needs also to store swap entries in the same tree: * entries in the same tree. They are marked as exceptional entries to
* those are marked as exceptional entries to distinguish them. * distinguish them from pointers to struct page.
* EXCEPTIONAL_ENTRY tests the bit, EXCEPTIONAL_SHIFT shifts content past it. * EXCEPTIONAL_ENTRY tests the bit, EXCEPTIONAL_SHIFT shifts content past it.
*/ */
#define RADIX_TREE_EXCEPTIONAL_ENTRY 2 #define RADIX_TREE_EXCEPTIONAL_ENTRY 2
#define RADIX_TREE_EXCEPTIONAL_SHIFT 2 #define RADIX_TREE_EXCEPTIONAL_SHIFT 2
static inline int radix_tree_is_internal_node(void *ptr) static inline bool radix_tree_is_internal_node(void *ptr)
{ {
return (int)((unsigned long)ptr & RADIX_TREE_INTERNAL_NODE); return ((unsigned long)ptr & RADIX_TREE_ENTRY_MASK) ==
RADIX_TREE_INTERNAL_NODE;
} }
/*** radix-tree API starts here ***/ /*** radix-tree API starts here ***/
@ -236,8 +245,7 @@ static inline int radix_tree_exceptional_entry(void *arg)
*/ */
static inline int radix_tree_exception(void *arg) static inline int radix_tree_exception(void *arg)
{ {
return unlikely((unsigned long)arg & return unlikely((unsigned long)arg & RADIX_TREE_ENTRY_MASK);
(RADIX_TREE_INTERNAL_NODE | RADIX_TREE_EXCEPTIONAL_ENTRY));
} }
/** /**